Table of Contents

Reference Architecture

2 Supermicro TwinPro Server Architecture

Supermicro 2U 4-Node TwinPro2
Reference Architecture for EsgynDB®

3 EsgynDB Enterprise Overview

Delivering on Converged Big Data Platform

2 Introduction
2 Solution Overview

4 Big Data Workloads

Executive Summary

5 Internet of Things (IoT)
5 Data Lakes
7 Solution Components
8 Supermicro Server platform
9 Reference architectures

Supermicro and Esgyn have teamed up to bring to market a Big Data Appliance thatdelivers faster business insights using EsgynDB, a Big Data converged database optimized
to run on the Supermicro 2U 4-node TwinPro2 SuperServer platform. This Big Data
Appliance comes complete with the software stack pre-installed, pre-configured, and pretuned for the use cases of Modernization of Data Lakes and Acceleration of IoT value, andto deliver the highest performance on standard benchmarks like YCSB, TPC-C and TPC-DS.
In fact, all 99 queries of the TPC-DS benchmark run straight out of box, and EsgynDB isthe only product on Hadoop that can do this. EsgynDB is tightly integrated with Hadoop
and supports HDFS, HBase, ORC, and Parquet. The Appliance is complete as-is or can beleveraged as a building block and scaled up to any cluster size needed.
Adoption of the Supermicro-Esgyn Big Data Appliance significantly reduces the complexity
in deploying a Big Data solution to modernize data lakes, to support hot and cold data
IoT analytics, and to create a converged database platform for future needs that include
support of multi-varied data and OLTP, ODS, BI and analytical workloads.
EsgynDB, with its mature ANSI SQL interface on Hadoop, running on the SuperServer
platform, can be used as a vehicle not only to rapidly execute pilot or proof of concept
projects, but also to clearly demonstrate acceleration of the ROI from Big Data investments.

Super Micro Computer, Inc.
980 Rock Avenue
San Jose, CA 95131 USA
www.supermicro.com

Target Audience: This document is intended for decision makers, system and solutionarchitects, system administrators and experienced users that are interested in reducing
design time or simplifying the purchase of a big data architecture containing bothSupermicro and Esgyn components. An intermediate knowledge of Apache™ Hadoop® andscale-out infrastructure is recommended.
Document Purpose: The purpose of this document is to describe the appliance bundle ofEsgynDB Enterprise on Supermicro cluster.

- March 2018

Introduction
Why Supermicro-EsgynDB
Appliance
The Big Data Appliance comes completewith the software stack pre-installed,
pre-configured, and pre-tuned, from theOS, to Hadoop, to EsgynDB, all tested, andcertified by Esgyn Labs, on a complete
Supermicro hardware platform that
includes servers, memory, storage, andnetwork adapters, all chosen for optimum
performance and reliability, targeting theuse cases of Modernization of Data Lakes
and Accelerating IoT vale.
It is sold as a building block unit withbuilt-in redundancy for high availability
and positioned as a single converged
platform for a spectrum of DBMS workloads
from online transaction processing (OLTP)
to operational data store (ODS) to data
warehousing (EDW) to business intelligence(BI) to online analytical processing (OLAP).
Available on free trial basis for up to 45
days.

This reference architecture has been created to assist in the rapid design and deploymentof EsgynDB Enterprise software on Supermicro infrastructure. It is also intended to identify
the software and hardware components required in a solution to simplify the procurement
process. The recommended Supermicro appliance has been carefully tested with a variety
of I/O, CPU, network, and memory configuration for transactional and analytical workloads.

Solution Overview
Supermicro and Esgyn solutions enable people to derive new business insights from Big
Data by providing a platform to store, manage and process data at scale. This referencearchitecture provides several performance-optimized configurations for deploying
EsgynDB Enterprise clusters on Supermicro infrastructure. When compared to previous
generations, the new configurations provide a significant reduction in complexity and arecognized increase in value and performance.
Configurations include preinstalled, pre-configured, and tested versions of Linux, Hadoop,and EsgynDB on the Supermicro server platform – a highly optimized server with theright mix of compute and storage performance characteristics for parallel processing
systems, like Hadoop. The configurations selected are the result of a great deal of testingand optimization done by Esgyn engineers resulting in the right set of software, drivers,
firmware and hardware to yield excellent workload performance characteristics.
Supermicro/Esgyn Big Data solutions provide excellent performance and high availability,with integrated software, services, infrastructure, and management – all delivered as
one tested configuration to significantly reduce deployment overheads and operating
expenses.

Supermicro TwinPro Server Architecture
The Supermicro TwinPro architecture is based on the Supermicro proven Twin technology
to provide exceptional throughput, storage, networking, I/O, memory and processing
capabilities in a 2U form factor. Customers can further optimize Supermicro solutions toresolve the most challenging IT requirements and benefit from exceptional Total Cost of
Ownership (TCO).
TwinPro solutions are designed for simplified deployment and maintenance, andassembled with the highest quality to ensure continuous operation even at maximum
capacity. Customers in high-end enterprise, data center, HPC and Cloud Computing
environments receive the greatest competitive advantage from data center resources with
the Supermicro TwinPro.

2••

Performance: The latest generation of Intel Xeon processors significantly heighten
performance of Supermicro TwinPro solutions. With up to 28 cores per socket, 6channels of DDR4 and up to 2TB of DDR4-2666 in 16 DIMM slots, memory speeds
will reach a maximum of 2666MHz. Supermicro X11 and X10 TwinPro systems are
optimized to deliver incredible performance to high performance environments andwill handle complex workloads with ease.

••

Flexibility: High flexibility and various networking options are key features ofSupermicro 2U 4-Node TwinPro2 Reference Architecture for EsgynDB®

Supermicro 2U 4-Node TwinPro2
Reference Architecture for EsgynDB®

the TwinPro. Each node includes a Supermicro I/O Module (SIOM) for networking
options such as 1G, 10G, 25G, 40G or 100G and 2 low-profile PCI-E 3.0 x16 expansion
slots, giving our customers the flexibility to include add-on cards when they are
required.
••

Efficiency: The Supermicro Twin Architecture product line delivers the industry's
densest, cost -efficient servers. The TwinPro doubles the performance of a traditional
2U system and houses four nodes to reduce data center footprint and includes
redundant Titanium Level Digital (96%) Power Supplies for maximum power
efficiency.

EsgynDB Enterprise Overview
The reference architecture leverages theSupermicro X10 generation TwinPro2
platform. It hosts 4 independent hot-swap
2-socket systems (nodes) in a 2U chassis.
The system is powered by redundant
2000W Titanium Level (96%+) power
supplies.

EsgynDB offers a unique value proposition by supporting the full spectrum of database
workloads at Big Data scale on a single converged database platform, simplifying yourdatabase landscape, reducing or eliminating data movement, and reducing overall latency.
EsgynDB supported workloads include:••

Online transaction processing (OLTP)••

Operational Data Store (ODS)••

Data Warehousing (EDW)••Business Intelligence (BI)••Online Analytical Processing (OLAP)

EsgynDB reduces ETL costs by transforming data (ELT) within the database. It’s Massively
Parallel Processing (MPP) architecture ensures that the strictest SLAs can be accomplishedby executing queries in parallel. A mature ANSI SQL engine enables application portability;
proven to handle petabytes of data at high concurrency in numerous demanding
enterprises for over 20+ years.
Node view (1 of 4)
Here is an overview of EsgynDB capabilities:••

Comprehensive and full-functioned SQL support allows companies to reuse andleverage existing SQL skills to improve developer productivity••

Easy integration with other ecosystem components to extend the data sources
available from streaming sub-systems to sentiment analysis and so on••

Support for hot and cold data••

Join structured, semi-structured, and unstructured data from disparate data sources
all within one SQL query••Interoperability with new or existing applications and 3rd party tools via support forstandard ODBC and JDBC••

Provides full ACID transactional protection across multiple rows, tables, and SQL
statements••

Sophisticated and massively parallel database engine supports complex queries athigh concurrency and throughput

Intel Inside ®. Powerful Productivity Outside.

Reference Architecture - March 20183

••World class Optimizer with abilities to tune the degree of parallelism to suit
individual queries and the scale or cardinality of data being queried••

Supports full active-active distributed transactions across data centers to scale
workloads with zero lost transactions for disaster recovery (EsgynDB Advanced
edition feature)••

Supports multi-tenancy to best optimize the available resources to meet SLAs ofdifferent tenants (EsgynDB Advanced edition feature)

This breadth of capability, referred to by Gartner as Hybrid Transactional and Analytical
Processing (HTAP) and by Forrester as Translytics, makes EsgynDB ideally suited to a range
of use cases, including:••Internet of Things (IoT)••

Data Lakes

Big Data Workloads
Typical Hadoop workloads can be broadly categorized into 4 different types: Batch, NonInteractive, Interactive, and Operational. These categories vary greatly in terms of their
response time expectations, concurrency, as well as the amount of data that is processed.
Hadoop is well positioned to address BI and analytical workloads. Supporting Operational
workloads on Hadoop is an emerging Hadoop market category and Esgyn is the only
company that can do this. Traditionally these workloads have been relegated to thedomain of traditional relational databases. But there is growing interest and pressure toembrace these workloads on Hadoop due to Hadoop’s benefits of significantly reduced
costs, reduced vendor lock-in, and its ability to seamlessly scale to larger workloads anddata sets.

4

Supermicro 2U 4-Node TwinPro2 Reference Architecture for EsgynDB®

Supermicro 2U 4-Node TwinPro2
Reference Architecture for EsgynDB®

Internet of Things (IoT)
Most IoT systems deal with real-time or near real-time data. Analysis of real-time data
together with older data on a single platform enables well-informed responses. In
technical terms, this could be construed as combining hot data with warm, and even cold
data, on a single platform. If this can be accomplished with minimal data movement anddata transformation, then the overall response times could be significantly reduced. This
results in a system that allows not only massive ingestion but also real-time reportingat the same time, and at significantly lower cost. The platform needs to deliver many
critical aspects such as scalability, availability, and high concurrence. EsgynDB can doexactly that, maintain sustained ingestion of transaction protected incoming data while
concurrently reporting or alerting on that same data. No batch updates. No data latency.
The Supermicro-Esgyn Big Data Appliance makes scaling out an IoT solution simple.

Data Lakes
Pumping any and all data into data lakes has not worked. Most of the data lakes are dry
or at best ponds or swamps as Business Units are not willing to contribute their data. Thisis because they are not able to see the same business reports with SLAs they are used
to. Currently their world is used to SQL interfaces for pre-canned or ad hoc queries, andself-served Business Intelligence reports. Even after allocating significant resources (Javaprogrammers, data scientists, Hadoop experts, or people with vendor-specific skills) data
lakes are not delivering to the expectations of business users. The promise of Hadoopbased data lakes may have been oversold and over purchased. Enterprises are pondering
issues such as: whether a data lake itself has become a silo instead of breaking down silos;
whether data lakes are moving applications away from data; or whether schema flexibility
is really helping to make life easier for business users. In almost all implementations, lackof a mature SQL engine is the common denominator why enterprises are not able to see
the promised return on investment. Adoption of the Supermicro-Esgyn Big Data Appliancesignificantly reduces the complexity in deploying a Big Data solution to modernize
data lakes. Get real value and actionable insights from your data lake by bringing theworkloads to the data lake. Most other database engines on Hadoop lack the maturity andsophistication to serve the full spectrum of workloads needed to maximize the potentialof your data lake. From operational to analytical, EsgynDB is the one database engine you
need. The Supermicro-Esgyn Big Data Appliance makes building a data lake easier andmaximizes the return on that investment.

EsgynDB innovations built upon Big Data software stackEsgynDB is designed to build upon and leverage Apache Hadoop and HBase core modules.
Operational applications using EsgynDB transparently gain Hadoop’s advantages ofaffordable performance, scalability, elasticity, availability, etc. Figure 1 depicts a subset of
the Hadoop software stack. Items in green are specifically leveraged by EsgynDB, such as
HBase and HDFS. To this stack, EsgynDB adds (items in orange) ODBC/JDBC drivers, theEsgynDB database software, and a distributed transaction management (DTM) subsystem
for transaction protection.
EsgynDB delivers innovation on top of Hadoop in these key areas:••

Intel Inside ®. Powerful Productivity Outside.

A full-featured ANSI SQL implementation whose database services are accessible via

Reference Architecture - March 20185

a standard ODBC/JDBC connection••

A SQL relational schema abstraction••

Distributed ACID transaction protection••

Full cross data-center active-active distributed transaction support, to scale reads
and writes, support local access and comply with safe harbor rules, with zero losttransactions in a disaster••

Performant response times for transactions comprised of both reads and writes••

Parallel optimizations for both transactional and operational reporting workloads

Figure 1.

6EsgynDB Ecosystem

Supermicro 2U 4-Node TwinPro2 Reference Architecture for EsgynDB®

Supermicro 2U 4-Node TwinPro2
Reference Architecture for EsgynDB®

Solution Components
Cluster Configuration
A number of key factors were considered prior to designing EsgynDB Big Data appliance.
The following subsections articulate the design decisions in creating the baseline
configurations for the reference architectures.

Operating System
Red Hat® Enterprise Linux®

Computation
When sizing the processing power for an EsgynDB Enterprise Supermicro cluster,
concurrency and types of workloads are the other key considerations for number of nodes.
The number of nodes and cores reflects the amount of parallelism available for concurrent
users of the applications running on the cluster. If typical workloads are high-concurrency
short queries, then nodes with fewer number of cores might be acceptable. If typical
workloads involve large scans, then more processing power is needed. Understand thetypes, frequency, plans, and typical concurrency for the application, ideally via prototyping
the workloads and queries whenever possible.

MemoryWhen sizing an EsgynDB Enterprise Supermicro cluster for memory usage, keep in mind
that many Hadoop ecosystem processes are Java processes. Due to memory efficiency
optimizations for the JVM, there is a significant restriction just below 32GB. Crossing thisthreshold results in less usable memory because the internal representation of pointerschanges in a way that consumes significantly more space. Plan for these processes to usea heap size of 16-32GB each for optimal performance on a large cluster. Reducing thememory for these components affects performance significantly, so do careful tuning andanalysis before choosing a smaller value. The primary users of memory in the EsgynDB
database engine servers based on query concurrency and workloads.

Storage
When sizing an EsgynDB Enterprise cluster for disk usage, for data nodes, SSD is only
beneficial for high concurrency write. However, HDD drives are sufficient, HDD data disks
configure disks as direct attached storage in a JBOD (Just a Bunch of Disks) configuration.
RAID striping slows down HDFS and reduces concurrency and recoverability. For control
nodes, data disks can be configured as either JBOD or RAID1 or RAID10.
As with processing power, disks are a unit of parallelism. For a given total-disk-per-nodevalue, if workloads include many large scans, it is often most effective to have more
smaller disks than fewer larger disks per node on data nodes. The reference architecture
assumes that most workloads include large scans. HBase SNAPPY or GZ compression isstrongly suggested. SNAPPY has less CPU overhead, but GZ compresses better. Degree ofcompression varies widely depending on the data and workload patterns, but generally
accepted calculations suggest around a 30%-40% reduction, depending on data.

Intel Inside ®. Powerful Productivity Outside.

Reference Architecture - March 20187

Compression adds to the path length for reading and writing, which can have an effect ondata growth and ingest. Compression happens at the HBase file block level, limiting theamount of un-compression required at read time. When calculating overall disk space anddata disk space per node, be sure to account for working space and anticipated ingest/
outflow per node. Also remember that blocks of an HDFS file come with a replication factor(typically set to 3, so 3 copies of the data). That means that each 10 GB file occupies 30GB
on disk. Esgyn recommends leaving approximately 33% of disk space free for overhead
workspace.

Network
In general, 10GigE is the standard for networking for data traffic within an EsgynDB cluster.
Using a slower network for data flow can significantly impact performance. 2 bonded
10GigE networks provide more throughput for I/O intensive applications.

Supermicro Server platform
The Supermicro TwinPro2 shown in Figure 2 below, is an excellent choice as the server
platform for the edge node, management node and head nodes.

Figure 3.

8

1 of 4 dual-processor nodes(all pictures are for illustration
purposes only)

Figure 2.

Rear View of Supermicro 2U TwinPro2 SYS-2028TP-HC0R-SIOM

Table 1.

Technical Specifications of Supermicro 2U TwinPro2 SYS-2028TP-HC0R-SIOM

Processor Support

Intel® Xeon® processor E5-2600 v4/v3 family (up to 145W TDP)

ChipsetIntel® C612 chipset

Memory Support

Up to 2TB ECC 3DS LRDIMM , up to DDR4- 2400MHz ; 16 DIMM
slots per node

Expansion

1 PCI-E 3.0 x16 (LP), 1 PCI-E 3.0 x8 (LP), 1 SIOM card support

Onboard Storage Controller

Broadcom 3008 SAS3 controller (8 ports); RAID 0, 1, 10

Drive Bays

6 Hot-swap 2.5" SAS/SATA drive bays

Connectivity

SIOM Networking

Power Supply

2000W Redundant Power Supplies Titanium Level (96%)

Supermicro 2U 4-Node TwinPro2 Reference Architecture for EsgynDB®

Supermicro 2U 4-Node TwinPro2
Reference Architecture for EsgynDB®

Network Configuration
Dual port 10G Ethernet network card: dual-port 10Gb Ethernet with SFP+ per node

Reference architectures
The following sections illustrate a reference progression of Hadoop clusters from a singlerack to a multi-rack configuration.

Single 2U TwinPro2 Architecture
The single 2U Supermicro EsgynDB Enterprise Reference Architecture is designed toperform well as a single-rack cluster design but also form the basis for a much larger multi2U design. When moving from the single-2U to Multi-2U design, one can simply add racks
to the cluster without having to change any components within the single-2U.

Multiple 2U TwinPro2 Architecture
The multi-2U design assumes the single unit design is already in place and extends its
scalability. The single unit configuration ensures the required amount of management
services are in place for large scale out. For multi-2U clusters, one simply adds extension
racks of a similar configuration to the single-rack configuration.

Performance Notes
Supermicro EsgynDB appliance delivers performance, scale and stability, which enables
customers to deploy Supermicro clusters with multi-2Us to run millions of queries, and athigh concurrency. Standard benchmarks like YCSB, TPC-C and TPC-DS results showed thatEsgynDB is the only mixed-workload engine that can consistently run OLTP, ODS, BI andanalytical workloads.

Esgyn Cluster Tuning/Optimization
Optimizer technology represents one of EsgynDB’s greatest sources of differentiation
versus alternative SQL on-Hadoop projects or products. Two primary areas are theextensible nature of the optimizer to adapt to change and add improvements; and thesophistication and maturity level of the optimizer to choose the best optimized plan for
execution.

Intel Inside ®. Powerful Productivity Outside.

Reference Architecture - March 20189

About Super Micro Computer, Inc.
Supermicro® (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology is a premier provider ofadvanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and Embedded Systems
worldwide. Supermicro is committed to protecting the environment through its “We Keep IT Green®” initiative and provides customers
with the most energy-efficient, environmentally-friendly solutions available on the market.
Learn more on www.supermicro.com
No part of this document covered by copyright may be reproduced in any form or by any means — graphic, electronic, or mechanical,

About Esgyn
Esgyn is the leader in Converged Big Data solutions that empower global enterprises to realize the potential of Big Data. With theindustry’s most mature, scalable and adaptive SQL Database for Big Data, Esgyn is leading the way enterprises cope with ever increasingdata management needs on-premises and in the cloud.
United States of America (USA)691 S. Milpitas Blvd., Suite 117
Milpitas, CA 95035 • +1-408-800-6916.
info@esgyn.com
Learn more on www.esgyn.com
Peoples Republic of China (PRC)
Beijing, Shanghai, Guiyang
PRC • +1-86-21-5082-2117
Learn more on www.esgyn.cn

including photocopying, recording, taping, or storage in an electronic retrieval system — without prior written permission of the
copyright owner.
Supermicro, the Supermicro logo, Building Block Solutions, We Keep IT Green, SuperServer, Twin, BigTwin, TwinPro, TwinPro², SuperDoctor
are trademarks and/or registered trademarks of Super Micro Computer, Inc.
Esgyn and EsgynDB are registered trademarks of Esgyn Corporation.
Ultrabook, Celeron, Celeron Inside, Core Inside, Intel, Intel Logo, Intel Atom, Intel Atom Inside, Intel Core, Intel Inside, Intel Inside Logo, IntelvPro, Itanium, Itanium Inside, Pentium, Pentium Inside, vPro Inside, Xeon, Xeon Phi, and Xeon Inside are trademarks of Intel Corporation orits subsidiaries in the U.S. and/or other countries.
All other brands names and trademarks are the property of their respective owners.
© Copyright 2018 Super Micro Computer, Inc. All rights reserved.
Printed in USA

Please Recycle

ESGYN_180118_Rev09