Patent ID: 12242359

DETAILED DESCRIPTION

The following description is provided to enable any person in the art to make and use the described embodiments. Various modifications, however, will be readily-apparent to those in the art.

Embodiments provide tenant-level recovery within a native multi-tenant database instance. Embodiments may provide such recovery even in cases where the native multi-tenant database instance does not provide tenant-level backup processes.

Recovery of a tenant of a first multi-tenant database instance to a particular point in time may begin with creation of a temporary database instance. The temporary database instance may be configured similarly to the first database instance and may include no tenant data or metadata. A backup (e.g., snapshot) of the first database instance which corresponds to the point in time is selected and used to recover the temporary database instance to the state associated with the backup. Log files may be replayed at the temporary database instance to bring the state of the temporary database instance to the desired point in time.

Next, data associated with the tenant to be recovered is exported from the temporary database instance. The data currently associated with the tenant in the first database instance is copied to external storage and then truncated from the first database instance. The data exported from temporary database instance, which is associated with the particular point in time, is loaded into the first database instance. The temporary database instance may then be removed. If errors are detected, the recovery operation may be rolled back by loading the data copied from the first database instance to external storage into the first database instance.

FIG.1is a block diagram of system100according to some embodiments. The illustrated elements of system100and of all other architectures depicted herein may be implemented using any suitable combination of computing hardware and/or processor-executable program code that is or becomes known. Such combinations may include one or more programmable processors (microprocessors, central processing units, microprocessor cores, execution threads), one or more non-transitory electronic storage media, and processor-executable program code. In some embodiments, two or more elements of system100are implemented by a single computing device, and/or two or more elements of system100are co-located. One or more elements of system100may be implemented as a cloud service (e.g., Software-as-a-Service, Platform-as-a-Service) using cloud-based resources, and/or other systems which apportion computing resources elastically according to demand, need, price, and/or any other metric.

Database platform110provides infrastructure for creating, managing and using native multi-tenant database instances. Database instance112provides native multi-tenancy according to some embodiments. Database instance112may be provisioned on any suitable combination of hardware and software, including one or more computer servers or virtual machines. In some embodiments, database instance112comprises a containerized application executing within a software container. Such containers may be implemented by one or more nodes of a cluster (e.g., a Kubernetes cluster) as is known in the art.

Database instance112includes execution engine114for responding to client queries based on data and metadata stored in volatile (e.g., Random Access) memory115. Memory115includes data1151and metadata1152of all tenants created within database instance112. Data1151may include row store tables, column store tables, and system tables. As is known in the art, the data of each row of a row store table is stored in contiguous memory locations of memory115, and the data of columns of column store tables is stored in contiguous memory locations of memory115. The system tables may store metadata defining a database catalog, users, etc. Memory115also stores program code and stack, and memory required for temporary computations and database management.

Multi-tenant application130may comprise a SaaS application but embodiments are not limited thereto. Multi-tenant application120may be provisioned on one or more computer servers or virtual machines and may comprise a containerized application executing within a software container. Multi-tenant application130issues queries (e.g., SQL, MDX) to database instance112based on input received from users142and147of customers140and145, respectively.

Database platform110according to some embodiments supports requests for tenant-level database operations which would otherwise need to be implemented by the application. These operations may include tenant creation, tenant drop, tenant move, tenant restore from backup, tenant clone, tenant resize and tenant resource limitation. In some embodiments, shared tenant service111exposes APIs (e.g., via REST) which are called by multi-tenant applications (using appropriate management credentials) to request these tenant-level operations from the database system using, for example, an associated tenant id. Current database system DDLs may be extended to support the assignment of database artefacts to tenants.

Each tenant of system100will be described as corresponding to a customer, where a customer may be a company, a division, a workgroup, or any other group of users. A tenant may correspond to a particular cloud resource/service subscription of a given customer. In this regard, a customer may be associated with more than one subscription and therefore more than one tenant.

Data115includes multiple instances of a tenant object defined in metadata1152. Each tenant instance is a collection of database artifacts, where the artifacts assigned to each tenant instance are stored within data1151. The database artifacts assigned to a tenant instance may include, for example, one or more schemas, tables, and partitions. The database artifacts may also include metadata defining views on the tenant's tables, virtual tables, caches, remote sources, workload classes used to govern resource usage for the tenant's database objects, and database users.

Memory115includes tenant instance1153of tenant ‘A’ and tenant instance1154of tenant ‘B’. Each tenant instance1153and1154is an instance of a tenant object defined in metadata1152. Tenant instances1153and1154may be stored within data1151but are depicted separate therefrom for ease of understanding.

Each tenant instance1153and1154is a collection of database artifacts. The artifacts assigned to each tenant instance1153and1154are stored among data1151and metadata1152. Accordingly, artifacts A1155assigned to tenant instance1153and artifacts B1156assigned to tenant instance1154are depicted using dashed lines to represent references to database artifacts of data1151and metadata1152. The database artifacts assigned to a tenant instance may include, for example, one or more schemas, tables, and partitions. The database artifacts may also include metadata defining views on the tenant's tables, virtual tables, and remote sources.

The lifecycle of a tenant may be decoupled from the lifecycle of its assigned database artifacts. However, in some embodiments, dropping of a tenant from a database instance results in dropping of artifacts assigned thereto, so long as those artifacts are not assigned to another tenant of the database instance.

It will be assumed that customer A140corresponds to a first tenant (e.g., tenant A1153) of database instance112and that customer B145corresponds to a second tenant (e.g., tenant B1154) of database instance112. Upon receipt of input from a user142of customer A140, multi-tenant application130may transmit a query to database instance112which indicates an association with the first tenant. Similarly, upon receipt of input from a user147of customer B145, multi-tenant application130may transmit a query to database instance112along with an indication that the query is associated with the second tenant.

Accordingly, multi-tenant application130is able to determine the tenant which corresponds to a user from whom input is received. For example, each user may logon to multi-tenant application130using a tenant-specific subscription. Multi-tenant application130therefore associates a user with the tenant of the subscription under which the user has logged on. In another example, communications between users and multi-tenant application130may include tenant-identifying tokens.

Multi-tenant application130is also aware of which tenants are placed on which database instances. In this regard, multi-tenant application130may request provisioning of database instances on platform110using a suitable service thereof and creation of tenants on provisioned database instances using tenant service111. Upon receiving input from a user associated with a given tenant, multi-tenant application130is thereby able to determine the database instance which includes the given tenant and to which a corresponding query should therefore be directed.

Upon receipt of a query from multi-tenant application130, execution engine114processes the query using the artifacts (e.g., row store tables) which have been assigned to the particular tenant with which the query is associated. Each time a query received from an application consists of a transaction on data in memory115, the transaction is logged as a log entry of a log segment stored within data1151. The pre-transaction version of the data page is stored as an undo data page, and the data page as changed by the transaction is marked as “dirty”. Periodically, and as is known in the art, a savepoint is created by writing the dirty data pages and the corresponding undo data pages of data1151to persistent storage120.

Persistent storage120includes data volume122for storing the data pages of the savepoint and log volume126for storing the log pages of the savepoint. The pages of the savepoint represent a consistent state of data1151of all assigned tenants. Backups1224comprise “snapshots” or saved copies of the data pages of respective savepoints, and backups128comprise snapshots of the log entries of respective savepoints. Accordingly, as is known in the art, data1151and metadata1152may be recovered to the state in which they existed at a particular point of time using the data backup134associated with a savepoint which was closest and prior to the point in time and entries of the log backup128which arose from the time of the savepoint to the point in time. This recovery would recover the data and metadata of all tenants to the particular point in time.

Persistent storage120may be implemented using any persistent data storage system that is or becomes known, including but not limited to distributed data storage systems. Persistent storage120persists encrypted data of all assigned tenants.

Database instance112thereby provides a single data server including the data and metadata of all tenants of the database instance, engines for processing the data, and a single persistence for the data and metadata. Hosting multiple independent tenants on such a single database instance facilitates sharing of computing resources at near-zero marginal cost.

Applications120and125may comprise SaaS applications but embodiments are not limited thereto. Applications120and125may be provisioned on a same or different one or more computer servers or virtual machines and may comprise containerized applications executing within a software container.

FIGS.2A and2Bcomprise a flow diagram of process200to implement tenant-level resource isolation according to some embodiments. Process200and all other processes mentioned herein may be embodied in program code executable by one or more processing units (e.g., processor, processor core, processor thread) and read from one or more of non-transitory computer-readable media, such as a hard disk drive, a volatile or non-volatile random access memory, a DVD-ROM, a Flash drive, and a magnetic tape, and then stored in a compressed, uncompiled and/or encrypted format. In some embodiments, hard-wired circuitry may be used in place of, or in combination with, program code for implementation of processes according to some embodiments. Embodiments are therefore not limited to any specific combination of hardware and software.

It is assumed that more than one tenant has been provisioned in a database instance prior to process200. According to some embodiments, provisioning of a tenant includes calling a tenant management service of a database platform under a tenant manager user account to create a tenant object instance (i.e., a tenant). The tenant object instance is associated with a tenant ID which represents the tenant. Typically, a database user account is then created and is associated with the tenant. More than one database user account may be associated with a single tenant.

Database artifacts are assigned to the tenant. Some embodiments may use a deployment infrastructure to define database artifacts to assign to a tenant. The deployment infrastructure may provide a declarative approach for defining database objects (e.g., as design-time artifacts) using containers. For example, a service broker may create a container which corresponds to a database schema and additional metadata (e.g., user, roles, privileges) which may then be assigned to a tenant instance. Such containers may be isolated from each other by schema-level access privileges.

Accordingly, prior to process200, a database instance such as database instance112including two or more tenants may be provisioned. The database instance may have operated for some time prior to process200to receive queries from a multi-tenant application. The multi-tenant application may have transmitted the queries based on instructions received from customers of the two or more tenants. Each query received by the database instance therefore identified a corresponding tenant and, in response, the database instance executed operations on the artifacts assigned to the corresponding tenant.

An instruction is received at S205to recover a first tenant of a first database instance to a prior point in time. The instruction may specify an ID of the database instance, an ID of the tenant, and the point in time. The instruction may be issued via a multi-tenant application by a user authorized to access a tenant service such as tenant service111. In some embodiments, the user authenticates to an authentication system using any suitable authentication protocol (e.g., multi-factor authentication), receives an authentication token in return, and passes the token to the tenant service.

In response to the instruction, a second database instance is provisioned in the database platform at S210. The second database instance may include the same configuration settings as the first database instance but no tenants or associated database artifacts.FIG.3illustrates database platform110including second database instance152created at S210in the present example. Database instance152includes execution engine154, allocated memory155and persistence160. In some embodiments, database instance152includes non-tenant-specific data and metadata in memory155upon creation, such as data and metadata needed or preferred for operation of database instance152.

Next, at S215, a backup (e.g., snapshot) of the first database instance which corresponds to the point in time is used to recover the second database instance to the state associated with the backup. S215may therefore include identification of backup124which is nearest and prior to the designated point in time and backup128which includes log files from the time of the identified backup to the designated point in time.

As is known in the art, memory155of database instance150is loaded with the data and metadata of the identified backup124during recovery at S210. Next, the identified log files of backup128are replayed at database instance150to bring the state of database instance150to the state in which database instance120existed at the designated point in time.

FIG.4illustrates database instance150according to the present example. Data′1551and metadata′1552represent data1151and metadata1152of database instance110as data1151and metadata1152existed at the designated point in time. Accordingly, data′1551and metadata′1552may differ from current data1151and metadata1152. Artifacts A′1155and artifacts B′1156, which are included within data′1551and metadata′1552, therefor similarly represent artifacts A1115and artifacts B1116as they existed in database instance110at the designated point in time. Database instance150therefore includes the data and metadata of each tenant stored in database instance120at the designated point in time.

At S220, artifacts of the first tenant are exported from second database instance to external storage. Export may consist of two operations-a first operation to export catalog information (i.e., metadata) describing tenant itself and its associated database artifacts and a second operation to export data of associated tables belonging to the tenant. The external storage may exist on the database platform (such as a separate storage container) or may be external thereto, such as a third-party cloud storage solution. In the present example it is assumed that the tenant to be recovered is tenant B. Therefore, as shown inFIG.5, the data associated with the tenant B (i.e., artifacts B′1556) is exported from data1551to external storage168.

The data currently associated with the first tenant in the first database instance is copied to external storage at S225. Database instance112may terminate receipt of queries associated with the first tenant prior to S225so that the copied data is in a consistent and known state. The external storage may be the same or different from the external storage to which the artifacts of the second database instance were exported at S220.FIG.5also shows the data associated with the tenant B (i.e., artifacts B1156) copied from data1151to external storage128.

The data currently associated with the tenant in the first database instance is truncated from the first database instance at S230. Truncation may be more efficient than deletion, since deletion deletes each row of each subject table and creates a transaction record corresponding to each deletion. In contrast, truncation simply deallocates the memory allocated to each table to be truncated.FIG.6illustrates database instance after truncation of artifacts B1156at S230. It should be noted that althoughFIG.6omits the dashed element representing artifacts B1156, the truncation occurs with respect to tables of data1151and metadata1152.

The artifacts of the first tenant exported to external storage are imported to the first database instance at S235. Importing the artifacts may comprise loading metadata to recreate all associated artifacts without recreating the tenant object itself, and then restoring the table data belonging to the tenant.FIG.7illustrates completion of S235, at which point memory115includes artifacts B′1156from external storage168. Since artifacts B′1156merely represent recovered data and metadata associated with the first tenant, memory115is further updated to illustrate data′1151and metadata′1152which include the recovered artifacts.

Next, at S240, the tenant recovery is validated. Validation at S240may include execution of a consistency check on artifacts B′1156or otherwise testing the recovered data. A customer of the recovered tenant may access the recovered data to assist in the validation. If the recovery is not validated for any reason, the copy of the artifacts stored at S225may be restored in the first database instance in order to roll back the recovery operation, and receipt of queries associated with the first tenant may resume.

Flow proceeds to S250if the recovery is validated at S240. At S250, the artifacts exported to external storage at S225and copied to external storage at S220are deleted. The second database instance is then deleted at S255.FIG.8illustrates architecture100after performance of process200according to present example. Architecture100ofFIG.8is identical to architecture100ofFIG.1but for artifacts B′1156, data′1151and metadata′1152which have been updated as described above.

Accordingly, upon resuming the receipt of queries associated with tenant B, database instance112will execute such queries on artifacts B′1556as they existed at the designated point in time. In contrast, artifacts A1155assigned to tenant A1153have not been rolled back and queries thereof are unaffected by the rollback.

Some embodiments encrypt all data prior to storage in persistence120. The data may be encrypted with customer-specific encryption keys. With reference toFIG.1, data122associated with artifacts A1155may be encrypted with a key specific to Customer A140while data122associated with artifacts B1156may be encrypted with a key specific to Customer B145. The encryption keys may be rotated from time-to-time, such that the key used to encrypt data122associated with artifacts B1156may be different from the key used to encrypt data122associated with artifacts B′1556from the prior point in time. Accordingly, after recovery of artifacts B′1556, the corresponding encryption key should also be rolled back to the encryption key specific to customer B145which was in use at the point in time.

FIG.9is a block diagram of system900providing native multi-tenancy and tenant-level encryption according to some embodiments. Cloud platform970includes database instance971. Database instance971stores data972within volatile memory at runtime as described above with respect to data1151. Volatile memory of instance971also includes symmetrical data encryption keys (DEKs), free memory, etc. Persistence973of instance971stores encrypted data as described above.

Customer A911includes key users912and business users913, and customer B917includes key users918and business users919. In some examples, a key user912may access multi-tenant application920to request provisioning of a database instance via cloud service broker940. Provisioning of database instance971may include generation of DEK DB985within persistence982of secure store980.

A tenant object instance may then be created in database instance950. Continuing the above example, a key user912may access multi-tenant application920to request creation of a tenant on database instance971. In response, tenant service965creates an instance of Tenant A based on a tenant object defined in metadata of data952. The instance of Tenant A may be identified by a tenant ID which is known to database instance971and multi-tenant application920. DEK A987is also created within persistence982.

A key user918of customer B917may also access multi-tenant application920to request creation of a tenant on database instance971. In response, tenant service965creates an instance of Tenant B in data972and DEK B989within persistence982. Multi-tenant application920further instructs tenant service965to assign artifacts to the tenant B instance.

After the provisioning of database instance971and creation of Tenants A and B, multi-tenant application920may, for example, receive input from a business user913of customer A911. In response, application920directs any resulting queries to database instance971along with an identifier of Tenant A. Database instance971therefore responds to the queries based on artifacts assigned to Tenant instance A. In a case that multi-tenant application920receives input from a business user919of customer B917, any resulting queries are directed to database instance971and responded to based on artifacts assigned to tenant instance B.

A key user993of database instance provider992provides KEK DB to key management system994for storage in key vault995. KEK DB is used to encrypt DEK DB985prior to storage thereof. Database instance971requests DEK DB from secure store980when database instance971wishes to decrypt tenant-unassigned data pages, such as during a restart process. In response, secure store980issues a request to key management system994to decrypt the stored encrypted DEK DB985using KEK DB996. Database instance971then uses the decrypted DEK DB985to decrypt the desired tenant-unassigned data pages.

Similarly, a key user912of customer A911provides KEK A to key management system994for storage in key vault995. KEK A is used to encrypt DEK A987prior to storage thereof. Database instance971may request DEK A987from secure store980in order to decrypt data pages of persistence973which are associated with Tenant A prior to loading thusly-decrypted pages into data952, or to encrypt data pages of data972which are associated with Tenant A prior to storing thusly-encrypted pages in persistence973. Store980issues a request to key management system994to decrypt the stored encrypted DEK A987using KEK A997. Database instance950then loads the decrypted DEK A989into its volatile memory and uses the decrypted DEK A989to decrypt the desired data of data955.

In some embodiments, secure store980polls key management system994to determine whether any KEKs have been revoked. Database instance971also polls secure store980to determine whether the KEKs of any of its tenants have been revoked and records such revocations. Accordingly, during the loading of a data page from persistence973to data972, it is determined whether a KEK required to decrypt the page has been revoked. If so, the data page is not decrypted or loaded, but its corresponding memory region is freed.

FIG.9also shows database instance975created according to process200in some embodiments. A backup of database system971is recovered to database instance975as described above, and the artifacts associated with a specific tenant are exported from database instance975to storage978provided by cloud platform970. The artifacts associated with the tenant in instance971are truncated, and the exported artifacts are loaded from storage978to instance971as described above. As also described above, it may be necessary to replace the current DEK of the tenant in persistence982with the DEK used to encrypt the imported artifacts.

FIG.10illustrates cloud-based database deployment1000according to some embodiments. User device1010may comprise any suitable computing system operable by a user to access a cloud-based application. User device1010may store and execute program code of a Web browser to access a Uniform Resource Locator (URL) associated with a login page of such an application. The Web browser may download and execute program code of a client-side component of an application as is known in the art.

Application server nodes1020,1022and1024may host an application according to some embodiments. The application may comprise a multi-tenant application and server nodes1020,1022and1024may be geographically distributed. Database nodes1030,1032and1034may host one or more database instances accessible to the multi-tenant application and providing native multi-tenancy as described herein. Database nodes1030,1032and1034may comprise an orchestration cluster and database cluster as is known in the art. Each node of deployment1000may comprise a separate physical machine or a virtual machine. Such virtual machines may be allocated by a cloud provider providing self-service and immediate provisioning, autoscaling, security, compliance and identity management features.

The foregoing diagrams represent logical architectures for describing processes according to some embodiments, and actual implementations may include more or different components arranged in other manners. Other topologies may be used in conjunction with other embodiments. Moreover, each component or device described herein may be implemented by any number of devices in communication via any number of other public and/or private networks. Two or more of such computing devices may be located remote from one another and may communicate with one another via any known manner of network(s) and/or a dedicated connection. Each component or device may comprise any number of hardware and/or software elements suitable to provide the functions described herein as well as any other functions. For example, any computing device may include a programmable processor to execute program code such that the computing device operates as described herein.

All systems and processes discussed herein may be embodied in program code stored on one or more non-transitory computer-readable media. Such media may include, for example, a DVD-ROM, a Flash drive, magnetic tape, and solid state Random Access Memory (RAM) or Read Only Memory (ROM) storage units. Embodiments are therefore not limited to any specific combination of hardware and software.

Elements described herein as communicating with one another are directly or indirectly capable of communicating over any number of different systems for transferring data, including but not limited to shared memory communication, a local area network, a wide area network, a telephone network, a cellular network, a fiber-optic network, a satellite network, an infrared network, a radio frequency network, and any other type of network that may be used to transmit information between devices. Moreover, communication between systems may proceed over any one or more transmission protocols that are or become known, such as Asynchronous Transfer Mode (ATM), Internet Protocol (IP), Hypertext Transfer Protocol (HTTP) and Wireless Application Protocol (WAP).

Embodiments described herein are solely for the purpose of illustration. Those in the art will recognize other embodiments may be practiced with modifications and alterations to that described above.