Cooperative model between an application server and a database

Methods and apparatus, including computer program products, are provided for implementing a cooperative model between an application and a database. In one aspect, there is provided a computer-implemented method. The method may include receiving from an application a command to perform an operation at a database; accessing metadata representative of the application; configuring, based on at least the metadata, a result of the operation; and sending the configured result to shared memory accessible by the application. Related apparatus, systems, methods, and articles are also described.

FIELD

The present disclosure generally relates to data processing.

BACKGROUND

Database systems are commonly employed in computing systems to store and organize information. A database system may include a database and at least one application program for accessing the database. A database is typically considered to be a self-describing collection of records. Each record may be a representation of some physical or conceptual object that contains information. The information contained in a record may be organized based on attributes. For example, if a database were used to keep track of employees in a corporation, each record might include attributes such as a first name, last name, home address, and telephone number. Records in a database are typically accessed using a key included in the database system.

SUMMARY

In one aspect there is provided a method. The method may include receiving from an application a command to perform an operation at a database; accessing metadata representative of the application; configuring, based on at least the metadata, a result of the operation; and sending the configured result to shared memory accessible by the application.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive. Further features and/or variations may be provided in addition to those set forth herein. For example, the implementations described herein may be directed to various combinations and subcombinations of the disclosed features and/or combinations and subcombinations of several further features disclosed below in the detailed description.

DETAILED DESCRIPTION

FIG. 1depicts a system100including an application110, which accesses via shared memory195a database190. The database190further includes a model115of the application110. The subject matter described herein relates to data exchanges between the database190and the application110based on the model115and/or shared memory195. The data exchanges are based on having (e.g., at, or accessible by, the database190) a model115of the application110, so that the data accessed (e.g., read, writes, copied, etc.) during the data exchanges are in a format directly usable by the application110. Thus, the database190has the model115for the application110and provides data in accordance with the model115.

The application110may be implemented as any type of application configured to handle data from database190. The application110may be implemented on a computer (which includes at least one processor and at least one memory) having access to database190. For example, the application110may be implemented as a browser, smart client application, and/or any other application. In addition, the application110may be configured to query data from database190and/or present data obtained from database190. AlthoughFIG. 1only depicts a single application, a plurality of applications may be used as well. When this is the case, model115may include a plurality of models to enable data exchanges according to each model.

The shared memory195may be implemented as at least one memory. For example, the shared memory195may be implemented in memory rather than using disk-based storage, such as electromechanical disk storage and optical disks. Examples of implementations of shared memory195include cache, dynamic random access memory (DRAM), static random access memory, and the like.

The database190may be implemented as any type of database. However, in some implementations, the database190is implemented as an in-memory database. Rather than using disk-based storage, an in-memory database keeps most, if not all, of the relevant database data in main memory, such as dynamic random access memory (DRAM), static random access memory, etc.

FIG. 2depicts a system200including a central processing unit210, cache212, main memory220, and persistent disk-based storage290. The following description refers toFIGS. 1 and 2.

In the implementation ofFIG. 2, main memory220further includes shared memory195. In addition, the database190(e.g., the database management system application for database190) and the relevant data for the database190are primarily maintained in main memory220. In this example, queries from the application110are performed on data in main memory220rather than on data in disk-based storage290. In some implementations, such use of main memory220reduces data access latency times by at least an order of about 200. Furthermore, the application110may also be hosted at system200. When that is the case, the data exchanged between application110and database190may be via shared memory195, which is contained within the main memory220used by both application110and database195, which may result in further reduction in data access latency times.

The database190may also be implemented as a column-oriented database, although a row-oriented database may be used as well. A column-oriented database refers to a database management system configured to store relevant data based on columns, not rows. On the other hand, a row-oriented database refers to a database management system configured to store relevant data based on rows, not columns.

FIG. 3depicts a table310including relevant data for country, product, and sales. In a row-oriented database, the relevant data is stored based on rows as depicted at row store320; while in a column-oriented database, the relevant data is stored based on columns as depicted at column store330.

Referring again toFIG. 1, in some implementations, system100may achieve faster data exchanges between the application110and the database190by also providing the model115. The model115may define the data types used by application110, the memory format/layout for data structures used by the application110, conversions of types or fields required by the application, and/or other format needs of application110. For example, if the application110is accustomed to operating with a row-based database, the model115defines the row structure used by the application110. On the other hand, if the application110is expecting a column-based database, then the model115defines the column-based requirement. In some implementations, system100may provide performance enhancements in one or more of the following: a reduced number of type conversions; a reduced number of library calls (in the application process); a reduced number of method calls; a reduced number of process changes (roundtrips); a reduced number of memory copies; and an improved memory management within the application.

At least one of the application110or the database190may configure data placed in shared memory195. The shared memory195may be used during the data exchanges between the application110and database190, although other communication mechanisms may be used as well. Moreover, the placement of data into shared memory195may be based on model115. For example, the format of the data placed in shared memory195may be in accordance with the requirements of application110as defined in model115.

During a read from database190, the application110may send a command, such as a SQL statement, to the database190. For example, the application110may read data from the database190using a SQL statement/command, such as select <field-list> from <table> command. The application110(or model115) may provide metadata describing the memory structure (e.g., layout) for the result of the command, such as the results of the SQL statement. For example, the metadata may define a line structure of a row table required by the application110, such that the result of the command, or SQL statement, is written to shared memory195in the structure defined by the metadata.

Next, the database190executes the command, such as the SQL statement. For example, a query may be executed at database190, which results in the generation of an internal query tree. Rather than sending the result of the query in a database-specific way to the application110, the database190materializes the result directly into the shared memory195(or some other type of shared buffer).

Because the result of the query of database190is written into shared memory190in a format specifically for the application110, the application110may accept the query result with little, or no, effort on the part of the application110(e.g., without further conversion). For example, in an implementation using ABAP at110, the application110copies a portion of the result from shared memory195into an address space in memory used by application110. In some implementations, the application110is R. The term R (which is available from http://www.r-project.org/) refers to a programming language and an environment/framework for providing an integrated suite of software components/functions for data manipulation, calculation, and graphical display, although mechanisms other than R may be used as well. For example, the application110may be the R programming framework executing an R script, such that the application110(which in this example is R environment) may access shared memory195when accessing and/or receiving information from the database. Specifically, a client library may integrate the result of the query into a data set by bypassing the R internal garbage collection and registering the data in shared memory195directly as if the data would be in a normal R address space.

When a large data exchange occurs, the application110may send massive amount of data to the database190. For example, application110may send a massive amount of data as part of a mass insert or a mass update and/or upset Rather than referencing in a SQL command the massive data amount field-wise (e.g., row-wise or column-wise), the application110may send a brief SQL statement to the database190, and the SQL command may separately reference the massive data amount being sent. In this example, the application110may provide metadata describing the memory layout of the additional data. This metadata may be handled and/or stored by model115. In implementations using ABAP at110, the metadata may include, for example, the line structure of a row table including low-level data types and offsets. In the case of R, the metadata are an implicit part of the data structure placed in shared memory describing R objects such as vectors or data frames (e.g., R table objects). The metadata resemble the internal data structures of R, which are complex data structures similar to special vector or table implementations. The application110then provides the data, which is used directly by the database190to execute the SQL statement.

At410, a command may be received from an application. For example, application110may send (either directly to database190or via shared memory195) a command, such as a SQL command to database190. The command may correspond to an operation at database190, such as a select statement or any other operation.

At420, metadata may be accessed representative of the application. For example, model115may provide metadata, such as definitions of the data types used by application110, the memory format/layout for data structures used by the application110, conversions of types or fields required by the application110, and/or other format needs of application110.

At430, the results of the operation are configured based on the metadata. For example, the results of a select statement performed on database190are configured in accordance with the accessed metadata, and then the configured results data is provided to (e.g., sent to application110and/or shared memory195). Examples of configuring data in accordance with metadata include converting data types or fields used by application110, configuring the memory format/layout for data structures used by the application110, writing data into shared memory in a format required by application110, and/or any other format needs of application110.

At435, the application access shared memory to obtain the configured results. For example, application110may access shared memory195to obtain the results configured in accordance with the metadata, such that the application110performs little, if any, conversion of the results configured in accordance with the metadata.

FIG. 5depicts a system500that is similar in some ways to system100including application model115and shared memory195coupled to application110and database190but system500includes additional elements as depicted atFIG. 5and described further below.

The application110couples to database190via a connection and session manager505. The connection and session manager505creates and manages sessions and connections for database clients, such as application110. For each session, the connection and session manager505maintains a set of parameters for the connection and/or session. Once a session is established, the application110may send commands, such as SQL statements, multidimensional expression (MDX) statements, and the like, to access database190.

A command received at database190is processed in the context of a transaction. The transaction manager510coordinates database transactions, controls transactional isolation, keeps track of running transactions, and keeps track of closed transactions. When a transaction is committed or rolled back, the transaction manager510informs those components of system100involved in the transaction so that the components can execute any necessary actions.

The commands received from application110are analyzed and executed by a request processing and execution control (RPEC) component520. The RPEC520includes a request parser for analyzing the command received from application110(e.g., a SQL statement and/or an MDX statement received from application110) and dispatches the command (or corresponding requests) to another element within database190. For example, transaction control statements are forwarded to the transaction manager510, data definition statements are dispatched to the metadata manager530, and object invocations are forwarded to object store540. Data manipulation statements may be forwarded to the optimizer522for creating an optimized execution plan, which is then given to the execution layer element524. The execution layer element524acts as the controller that invokes the different engines362A and362B and routes any intermediate results to another portion of the execution process.

The database190may also include domain-specific portions, such as a financial planning engine526D. In addition, the database190may include scripting, such as for example, SQL scripting526B that enables running application-specific calculations inside database190. The SQL script526B may be configured for optimizations and parallelization of SQL statements. RPEC520may also be configured to support multidimensional queries via MDX component526C.

The planning engine526C may provide financial planning applications to execute basic planning operations in the database190. For example, planning engine526C may allow creation of a new version of a data set as a copy of an existing data set while applying filters and transformations. In this example, planning data for a new year is created as a copy of the data from the previous year by filtering by year and updating the time dimension.

The SQL script526B, MDX526C, and planning engine526D are implemented using a calculation engine528, which provides a common infrastructure for SQL script526B, MDX526C, and planning engine526D.

Metadata may be accessed via the metadata manager530. The metadata may include a variety of objects. Examples of metadata include the definition of relational tables, the definition of columns, the definition of views at application110, the definition of indexes, the definition of SQL script functions, and object metadata. Metadata of all these types may be stored in one common catalog accessible by components of system500regardless of whether the store is in-memory row store, in-memory column store, object store, disk based, etc. The database190may also support multi-version concurrency control and multiple tenants, some of which may share common data and some of which may have separate tenant-specific data.

The database190may include one or more relational database engines360operative with in-memory row store362A and/or in-memory column store362B, although disk based databases may be used as well. The in-memory row store362A is a relational in-memory database engine that stores data in a row based way. The in-memory column store362B is a relational column based in-memory database engine. Although most, if not all, of the relevant data handled by database190is stored in-memory, some data may be stored on a disk-based storage device (e.g., data that has aged and is no longer in use or tracing data). Data in disk-based storage, such as disk storage290, is primarily stored on disk and only moved in-memory when accessed.

When a database table is created, database190specifies where it is stored, e.g., whether the table is stored in row, column, disk, or a combination of row, column, and disk. Moreover, database190may allow tables from different stores to be combined using one statement (e.g., a join, a sub query, a union, etc.).

As row-based tables and column-based tables may be combined in one SQL statement, the corresponding engines362A-B consume intermediate results cached in362D. A difference between the row-based and column-based database engines362A-B is the way the engines process data. For example, row store operators process data one row-at-a-time using iterators, and column store operations (such as scan, aggregate and so on) require that an entire column of data be available in contiguous memory locations. To exchange intermediate results at362D, row store362A can provide results to column store362B materialized as complete rows in memory, while column store can expose results using the iterator interface needed by row store362A.

The object store540is an in-memory store for graph-like structures represented by networks of objects. The object store540may be used to optimize and plan tasks that operate on large volumes of graph-like data, such as supply chain management data.

The data aging manager564is used to manage the movement of data from in-memory mechanisms to persistent, disk based storage290. For example, the data aging manager564may regard data as aged if it is no longer needed during normal business (which includes transactional processing as well as reporting use cases), if the data is rarely accessed, or a significant amount of time has passed since being accessed. When this is the case, the data aging manager564may move the data from memory to disk storage290.

The persistence manager592manages the durability and atomicity of transactions. For example, the persistence manager592ensures that the database190is restored to the most recent committed state after a restart and that transactions are either completely executed or completely undone. The persistence manager592also provides interfaces for writing/reading data, and contains a logger for managing a transaction log.

The database190may also include an authorization manager594, which is invoked by other components of system500to check whether a user has the required privileges to execute a requested operation. A privilege grants a right to perform a specified operation (such as create, update, select, execute, and so on) on a specified object (for example a table, a view, a SQL Script function, and so on). Users may also be authenticated via for example a login with user identification and password.

Although a few variations have been described in detail above, other modifications are possible. For example, while the descriptions of specific implementations of the current subject matter discuss analytic applications, the current subject matter is applicable to other types of software and data services access as well. Moreover, although the above description refers to specific products, other products may be used as well. In addition, the logic flows depicted in the accompanying figures and described herein do not require the particular order shown, or sequential order, to achieve desirable results. Other embodiments may be within the scope of the following claims.