PhD Thesis Review, No. 17

bottleneck. Moreover, queries were optimized andexecuted independently of each other in a query-at-a-timeprocessing model with few of the queries actually runningsimultaneously on the traditional platforms.
Nowadays the database systems are facing a different
environment where the modern multicore processors areproviding powerful parallel computing capabilities than
traditional uniprocessors. Recently micro-processor
manufacturers find that it has become increasingly
difficult to make CPUs go faster due to size, complexity.
So they continue the performance curve. That is, putting
multiple CPUs on a single chip and relying on theparallelism ability to get higher performance gain whichbrings the computing world into so-called “multi-core era”.
The multi-core processor is the mainstream now and there
will be many-core processors with more cores on one die.
As the processor speed increased as a much higher
speed than the development in the memory access speed,
the multicore processors are costing more cycles in
waiting intended data fetching from memory. The
performance of the memory subsystem is greatly affecting
the whole system for data intensive applications andunoptimized memory access pattern has the possibility to
slow
down
the
whole
system.
Microprocessor
manufactures have also recognized the data starving
problem on modern multicore platforms and are putting amuch bigger on-chip cache space to address thisperformance gap. Therefore, efficient cache utilization
and reducing the time consuming memory accessing
operation become critical for efficient DBMS on modern
multicore platforms. Optimizing cache performance often
requires very careful tuning or the total re-writing for thedata structures and algorithms. However, considering thecomplexity of modern commercial database engines, any
modification is a challenge and time consuming work.
Therefore, a more practical cache optimizing solution
which can avoid the challenge work of modifying existing
implementations of DBMSs will be welcomed andexpected.
The increasingly powerful concurrent processing
ability bought by multicore processors is stressing thedatabase engines. A traditional approach of getting higher
performance of processors is to increase the clock speed ofthem, since a faster CPU can finish one task quickly thenswitch to the next. In the multicore area, we can notbenefit from the increasing of clock speed anymore, but wehave to efficiently utilize the parallelism ability brought
by increasing of a core number instead. That is, thesoftware must have scalability and provide highparallelism to keep the multicore processor busy. The
original concurrency ability which is mainly used tooverlap delays in traditional DBMS cannot exploit heavy
parallelism. The database applications can exhibit
concurrently both inter and intra queries, and the
database engines perform well on the platform with lessthan 10 cores. For the sudden boost of modern multicore
and many core platforms with 40 to 100 hardware threads,
the existing database engines are clearly not yet ready.
Existing research pointed out some functions inside thedatabase engines are becoming into new bottleneck andprevent the system scale to a high level on the multicore
platform. Considering the complicity of DBMS, with

A Research on Efficient andScalableDBMS
forMany-core Based Platform
メニーコアプラットフォームのための効率的かつスケーラブルな DBMS に関する研究
Fang XI 1
In recent years, dramatic improvements havebeen made in computer hardware. In particular,
the number of cores on a chip has been growing
exponentially,
enablinganever-increasing
number of processes to execute in parallel. Having
been
developed
originallyforsingle-coreprocessors,
databasemanagement
systems(DBMSs) cannot take full advantage of theparallel computing that uses so many cores.
Therefore, this thesis analyzes the possibility of
optimizing the performance of DBMS on modern
multicore platforms and proposes two approaches
to break the bottleneck in database engines and toprovide cache-efficient query scheduling for
concurrent
queriesfordifferent
databaseapplications on modern multicore platforms. Allof these proposals are pure middleware, which
avoid any modification to existing DBMSs,
thereby making them more practical. This isimportant because the source code for existing
DBMSs is large and complex, making it very
expensive to modify. The performance evaluation
throughbenchmarks
revealedthat
these
proposals can improve throughput and scalability
on typical modern multicore platforms fordifferent database applications.	
 1. Introduction
Database management systems (DBMSs) are widelyused to help users and applications control their data overthe decades, by providing both efficient data update, data
retrieval and durable, structured data storage. Since thebirth of the DBMS, how to provide fast and timely data
access has become into the challenge for researchers andthe DBMS vendors. Most DBMSs were designed in the
1980s, when only uniprocessors were available andlimited main memory space. The data are stored on thedisks, and the main memory is used to buffer the usuallyaccessed data in order to reduce the time consuming disk
accesses. Research focused on improving the system
performance through efficient buffer pool management
and fine-grained multiplexing concurrent transactions tohide disk latency, as the disk I/O was the dominated___________________________________________________________________________________________

Fang XI（西方）received her Ph.D in Computer Sciencefrom Tokyo Institute of Technology. She is engaged inresearch on data engineering and database systems.
1

17 - 1

Database Society of Japan

PhD Thesis Review, No. 17

on different processor cores affect the performance ofprivate-cache levels, which are closer to the execution unit
than the LLC. We then propose a middleware-based
system to provide efficient data access to theprivate-cachelevelsfor
concurrent
OLTP-styletransactions on multicore platforms.

increasing the concurrency of transactions, there will bemany more potential bottlenecks exposed. For example,
the concurrent update to the same table will cause severe
contentions in the lock functions. Experiments to find thepotential bottlenecks and removing these bottlenecks
from database engines is very important for achieving
high scalability and high throughput.
Motivated by these challenges on modern multicore
and manycore platforms, this thesis analyzed thepossibility to provide efficient and scalable DBMSs
through optimizing the concurrent queries as a whole, andanalyzed how to employ sharing for concurrent queriesrather than optimizing each query independently for
different kinds of workloads.
We proposed the CARIC-DA (Core affinity with arange index for cache-conscious data access in a multicore
environment) [1] to optimize the high level data cacheperformance for OLTP applications on multicore
platforms. We dispatch the concurrent database queries torun on different cores according to the different dataneeds of the concurrent queries. By co-running the queries
with the same query needs onto the same processor core,these concurrent queries can share data in private cachelevels and reduces the cache miss and the time consuming
memory accesses.
Furthermore, with the other proposal of PM-DB(Partition-based multi-instance database system for
multicore platforms) [2], we solved the contention inmemory space management function of existing databaseengines for concurrent queries, by managing multiple
database engines on a single multicore platform. By
distributing the concurrent queries to different databaseengines, the original contention for a single engine can beeased and the whole system achieved higher scalability.

2.2 M otivation of CARIC-DA
A typical OLTP workload consists of a large numberof concurrent short-lived queries, each accessing a small
fraction of a large dataset. Furthermore, all concurrent
DB processes (DBPs) dealing with various queries should
be dispatched to run concurrently on different processor
cores. However, a different DBP-dispatch decision willlead to different cache performance. For example, Figure
1 shows two query-dispatching strategies for four
concurrent queries on two processor cores. In thecache-efficient solution (Schedule 1 in Figure 1), the twoqueries that access the same data are dispatched to run
on one processor core. Q1 can reuse the cache data <1-50>
which are already loaded into the cache after the
execution of Q2. However, in the cache-inefficient solution(Schedule 2), Q1 runs after Q3, and it cannot reuse thecache data loaded by Q3. There will be more cache misses
in the cache-inefficient solution than in the cache-efficientsolution. Furthermore, the cache-efficient solution can
restrict the data access for the private cache of each core
to within a smaller subset, with the probability of cachehits thereby being increased.

2. The CARIC-DA System

The data intensive applications of DBMSs are far
from taking full advantage of the parallel processing
capabilities provided by modern multicore platforms. Thisis largely due to the fact that the advance in the speed ofmulticore processors far outpaces that in memory latency,
leading to data cannot be delivered fast enough to be
consumed by the processor. The CPU cache, which is used
to buffer the main memory data blocks to speed up
accesses to frequently needed data, becomes critical forovercoming the “memory wall”.

Figure 1: Different query-dispatching strategies and related
private-cache performance

2.1 Related W orks
Ailamaki et al. analyzed the memory hierarchy
performance of commercial DBMSs and pointed out theimportance of the last-level cache (LLC) [3]. A study on
MCC-DB [4] pointed out the conflicts in the shared cachefor concurrent queries on the multicore platform.
As the cache levels become more complex and thelast-level cache size is scaled, the access to the LLC
involves an increasing number of clock cycles. Thesechanges in cache levels indicate that it is increasingly
important to bring data beyond the LLC and closer to L1.
Hardavellas et al. proposed STEPS [5], which minimizes
instruction misses in the L1 cache based on the StagedDB
design. However, reducing the data misses in higher
cache levels is still a major challenge.
Therefore, in this approach, we first analyze howvarious scheduling strategies for concurrent DB processes

Figure 2: CARIC-DA system on multicore platform

2.3 Fram ework of CARIC-DA
To provide efficient private-cache utilization for eachprocessor core, the queries that access the same dataset
should be dispatched to run on the same core. However,the DBMS cannot ensure that, as the scheduling of DBPs

17 - 2

Database Society of Japan

PhD Thesis Review, No. 17

database systems. Mixed workloads such as thosemodeled in the TPC-W benchmark [9] are different fromboth typical OLAP and OLTP applications. There areneither as many complex queries as in OLAP applications,nor as many severe update operations as in typical OLTP
workloads. To this end, the optimization of mixed
workloads on multicore platforms is still a challenge forexisting database engines.
We analyzed the scalability of the mixed workload
using the TPC-W benchmark on a modern Intel E7
multicore platform with 80 hardware contexts. Results
show that the system encounters severe scaling problems.
There are 80 hardware threads on the multicore platform,
although the system could only scale to 40 concurrent
clients. We further analyzed the CPU utilization for thesystem using the performance monitoring tool Perf. As thenumber of concurrent clients increases, the “s_lock
function” in PostgreSQL becomes the system bottleneck.
Each database engine holds one shared buffer in memory
and the concurrent loads access the shared memory space
through specific synchronization functions. The “s_lock”
function is the hardware-dependent implementation of aspin lock and it is used to control access to the sharedmemory-critical sections in PostgreSQL [10][11].
Therefore, we can conclude that the contention for theshared memory significantly degrades the scalability ofthe database engine on a multicore platform that could
otherwise offer rich true concurrency not provided on
single-core platforms. Considering the complexity of mostcommercial database engines, the rewriting of existing
database engines will be a very challenging task [12].
Therefore, we proposed the PM-DB, a partition-based
multi-instance solution that was originally used inshared-nothing parallel database systems. PM-DB
exploits the parallelism offered by multicore through the
combined performance of a collection of unmodified
database engines, rather than through the optimization ofa single engine modified to run on multicores.

on processor cores is decided by the OS. CARIC-DA offersa practical approach with no modification needed in either
the DBMS or the OS. It achieves its goal by a two-step
strategy.
(1) Dataset and DB-process binding: CARIC-DA
associates each DBP with a disjoint subset of the DB and
to ensure that queries that access data in the same subset
are executed by the same DBP.
(2) DB-process and processor-core binding: CARIC-DA
forces each DBP to run only on a specific processor core bysetting the CPU affinity for each DBP. The core affinity
setting is achieved via a function provided by the LinuxOS called the CPU affinity [6]. In this way, we reach thegoal, namely, binding between datasets and cores.
The architecture of the CARIC-DA system deployedon a multicore platform is shown in Figure 2. We extend
an existing DBMS in terms of a middleware level between
the DBMS and the clients. CARIC-DA offers a single
system image to the clients, and the clients do not need toknow about the data-partitioning information in the
database system. We introduce some middleware
processes to dispatch the queries to be processed bydifferent database processes according to the pre-defined
database partitioning information. Therefore, we canmake sure each processor core only deals with the queries
accessing the data in a specific sub data set.

3.1 Architecture Overview
The architecture of the PM-DB system deployed on asingle multicore platform is shown in Figure 4. We set up
several database instances rather than one on themulticore platform and partition the whole database into
the different database instances. The detailed database
partition information is stored in the local data structure
as a Range Index. Each database instance's processmonitors its Query Buffer, and takes the query from thebuffer when it arrives. After execution, the query result
will be written back into the query buffer and the Query
Dispatcher then returns the answer to the client.
For some applications, it will be difficult to provide aclean partition; that is, some transactions will require
data that are stored in different instances. The PM-DB's
Query Dispatcher function can dispatch the transactions
to several instances that hold the transaction required
data. Moreover, for the update transactions that access
multiple instances, we provide the two-phase commit
protocol to ensure that either all the instances are
updated or none of them; therefore, the databaseinstances can remain synchronized with each other. The
multicore-based two-phase commit has much lower
overhead
than
that
in
traditional
distributed

Figure 3: Performance of the CARIC-DA system

2.4 Experim ents
The CARIC-DA-related functions were implemented
in the C language over the PostgreSQL and the Linux. We
used the micro benchmark to isolate the effects, and thenwe used the more complex TPC-C. The response time andscalability of the CARIC-DA compared with the Baseline
system on different platforms are shown in Figure 3. The
proposal can reduce the miss ratio for the L2 cache forabout 50% on the AMD platform (48 cores) [7], while on
the Intel platform [8], the L2 miss ratio is reduced by 7%.
For the TPC-C benchmark with data set of 4.8GB on AMD
platform (40 cores/80 vCPUs), the CARIC-DA system canachieve 25% throughput improvement with 48 clients.

3. The PM-DB System

The increasingly powerful concurrent processing
ability of multicore platforms is stressing these DBMSs.
An increasing number of concurrent database processes
share resources both at the hardware (caches andmemory) and at the software (locks) levels, and any
inefficient resource sharing will create new bottlenecks in

17 - 3

Database Society of Japan

PhD Thesis Review, No. 17

environments, as the data communication is much fasterthrough the interconnections in the single multicore
platform than in network communications. For
applications with frequent crossing-instance join
operations, the data transfer between different instances
may slow down the transactions. Therefore, ourmiddleware can provide data duplication to avoid
intensive data transfers between instances byredundantly storing some data in several instances.

3.3 Perform ance Evaluation
We analyzed the efficiency of our proposed PM-DB
system on a modern Intel multicore platform using the
TPC-W benchmark. The hardware of the DB server is theIntel Xeon E7 system that serves as 80 virtual CPUs with
Hyper-Threading. By setting up more database instances,the contention in the single database engine can be
reduced and experiments show that our proposal achieved
at most 2.5 times higher throughput than the Baseline
system (Figure 5 (a)). Moreover, the cache-efficient querydispatching provided by the PM-DB can further improve
the system throughput by 21% (Figure 5 (b)).

4. Conclusion

In this thesis, I research on the performance andopportunity for providing an efficient and scalable DBMS
on modern multicore and many-core platforms. Two novel
approaches which are CARIC-DA and PM-DB are
proposed, for overcoming the processor memory gap andnon sacalable bottlenecks in database engine for OLTP
and mixed applications. Experiments show that the
CARIC-DA and PM-DB systems achieved higher
performance on both of modern AMD and Intel platforms.

References[1] F. Xi, T. Mishima, and H. Yokota, “CARIC-DA: Coreaffinity with a range index for cache-conscious data accessin a multicore environment,” DASFAA, pp. 282–296, 2014.
[2] F. Xi, T. Mishima, and H. Yokota, “ PM-DB:
Partition-based multi-instance database system for
multicore platforms,”ICEIS, pp. 128-138, 2015.
[3] A. Ailamaki, D. J. DeWitt, M. D. Hill, and D. A. Wood,“DBMSs on a modern processor: Where does time go?”
VLDB, pp. 266–277, 1999.
[4] R. Lee, X. Ding, F. Chen, Q. Lu, and X. Zhang,“MCC-DB: Minimizing cache conflicts in multi-coreprocessors for databases,” VLDB, pp. 373–384, 2009.
[5] S. Harizopoulos and A. Ailamaki, “STEPS towards
cache-resident transaction processing,” VLDB, pp. 660–
671, 2004.
[6] R. Love, “Kernel korner: CPU affinity,” Linux Journal,
no. 111, p. 8, July 2003.
[7]
BKDG
for
AMD
family10h
processors.
http://support.amd.com/en-us/search/tech-docs?k=bkdg.
[8] Intel 64 and IA-32 Architectures Optimization
Reference Manual.
http://www.intel.com/content/dam/doc/manual/64-ia32-architectures-optimization-manual.pdf.
[9] D. Menasce, “TPC-W: A benchmark for e-commerce,”
Internet Computing, IEEE, vol. 6, no. 3, pp. 83–87,
2002.
[10] R. Johnson, I. Pandis, N. Hardavellas, A. Ailamaki,
and B. Falsafi, “Shore-MT: A scalable storage man- ager
for the multicore era,” In EDBT, pp. 24–35, 2009.
[11] T. I. Salomie, I. E.Subasu, J. Giceva, and G. Alonso,“Database engines on multicores, why paral- lelize
when you can distribute?” In EuroSys, pp. 17–30, 2011.
[12] G. Giannikis, G. Alonso, and D. Kossmann,“SharedDB: killing one thousand queries with one
stone,” In VLDB, pp. 526–537, 2012.

Figure 4: PM-DB system deployed on a multicore platform

3.2 Cache Optim ization
We consider a modern multisocket and multicore
platform, and propose two strategies to manage therunning of the concurrent database processes on themulticore platform. First, we propose to run the databaseprocesses that access the data in different databaseinstances on different processors. This strategy can
avoid forcing queries that access data in different data
partitions to compete for Last Level Cache (LLC)
resources. Second, for the database processes in eachdatabase instance, we propose to separate different typesof queries to run on different processor cores. This
strategy can increase the performance of private cachelevels. If we run simple queries together on the sameprocessor core, there will be a higher possibility that these
queries can share high-level index data in the private
cache levels. However, if complex queries (with join
operations) run together with simple queries on the samecore, the one-time accessed hash data or a big range oftable data of complex query will evict the frequently used
index data of simple queries from the private cache.

Figure 5: Performance of the PM-DB system on Intel
platform

17 - 4

Database Society of Japan