Cloud Storage andEC solution example

田口栄治
クラウド・コンピューティング事業本部
インテル株式会社
Intel Confidential
2012年7月

• *Other names and brands may be claimed as the property of others.
• Copyright ©2012 Intel Corporation.

Erasure Coding (EC)RAID

Erasure Coding

AppServer

App

data

data

Meta
Data
Service

RAID 5/6

SAS toSCSI
to disks
disk
data
1

datamm data
Disk

Disk

data
m+1

ErasureCoding Client

SCSI to disk or IP to storage servers
datam+2

1 or 2 parity
Disk

data
location

Diskslice
1slicemm minimum
Storage
ServiceStorage
Nodeslicenslice
m+1

k spareStorageServiceStorage
ServiceStorage
ServiceStorage
NodeStorage
NodeStorage
NodeEC extends the data protection architectures of RAID 5/6 to RAID kk = the number of failures that can be tolerated without data loss:
For RAID 5, k=1; For RAID 6, k=2; For EC, k = n

EMC* Atmos* and Isilon* are example systems using EC• *Other names and brands may be claimed as the property of others.
• Copyright ©2012 Intel Corporation.

Optimized Object Storage
Traditional
Object Storage
Interface••

http/REST API provides objectaccessObjects names are user-specified

Server Mgmt.

Namespaces + RedundancyPolicies••

Virtual containers
Policies define protection level(e.g. 16/4) 16TB 16TB

FileSystemsSingle System16TBView••

Object Storage

2TB

2TB

Objects are globally accessible
No Volume Management

Object BasedInterfaces

PythonDynamic
Space 1NameSpaces

No volumeMgmt

RESTSpace xxxx..

Single Pool

VolumeGrid Architecture
2TB
2TB
•

16 TB

Loose any disk, server, rack ordatacenter

1 Large GridDisks.

Copyright 2011 Amplidata4

Core Technology Components• BitSpread – Distributed Encoder/Decoder••••

RAID replacement technology based on unique variant of Erasure Coding“Dial-in” fault tolerance through namespace level policies• Namespace1: 16/4 policy protects against any 4 failures in 16
disks/nodes• Namespace2: 18/6 policy protects against any 6 failures in 18
disks/nodes• Namespace3: 8/2 policy protects against any 2 failures in 8 disks/nodes• …
Provides availability and reliability even during failures
Policies can be dynamically changed• BitDynamics – Maintenance & Self-Healing
Agent••Out of band operations agent for disk monitoring, integrity verification &object self-healing
Performs automated tasks: scrubs, verifies, self-heals, repairs & optimizes
data on disk
Copyright 2011 Amplidata

Object Storage System Topology
End-UserApplications• BitSpreadAppApp• BitDynamics

HTTP

Cloud
http/REST

http/
REST
10GbE

Scale-out
IO controllers

Gigabit
Ethernet
FabricHigh-density,
low-power
Storage modules

Scalable
Storage
Pool

Copyright 2011 Amplidata

The BitSpread Algorithm – Simple Example••••

BitSpread Encodes data in unrecognizable chunks (actually a series of equations)
Distributes the equations across disks, storage nodes, racks, data centers
Original data can always be uniquely determined from a subset of the equations
BitSpread codec actually uses 4K variables & equations independent of object size
Simplified mathematics
Original Object

Decomposed Object

75

75

BitSpread

Series of Equations

X+Y=12Any 2 out of 3equations uniquelydetermine object

7

X-Y=275

2X+Y=195757

Large Object Store Rack(Storage Node)

Large object storagex2 1/10GbE BaseT switches

Amplidata*

960 TBs storagex40-1/10GE x810GE x8x40-1/10GE10GEX86-64 RedHat* Linux*

x32 storage servers

Client/MD Server
Client/MD Server

42 RU

Client/MD Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage Server
Storage ServerIntel®
206
chipset

DMI
G2x4 SATASuper Micro*
SASLP-MV8x6x10SATAIntel®
Xeon®
E3-1220Lx4
PCIeG2x4
PCIeG2Intel®
Dual port GbE

SAS 6Gb/s

x2
x32 GbE

6Gb/s

Hitachi*
3TB SATA
Storage Drive

2G ECC
DDR3Memoryx2 GbE

x2
Arista 7140T

x10 1u 3.5”
SATA Disk Enclosure

Storage Server Reference Architecturex2
x8 10GbE
SFP+~1PB of raw storage in a 42u rack
High Efficiency, Durability, Scalability with Erasure Coding• *Other names and brands may be claimed as the property of others.
• Copyright ©2012 Intel Corporation.

Converged Storage Server with EC Value(320 Drive, 960TB comparison, no single point of failure1)

Value

Efficiency

Durability

DescriptionNumber nodes=32, 10 drives/node, Cap/Node=30TB
EC16
m=10, k=6
16 nodesRAID0+1
m=10, k=0
2 nodes

RAID5+1
m=9, k=1
2 nodes

RAID6+1
m=8, k=2
2 nodes

RAID 3way
m=10, k=0
3 nodes

Raw/Usable
Efficiency

63%

50%

40%

34%

33%

Usable Capacity
(TB)

600

480

432

384

320

Watts/Usable
Capacity

53%

67%

74%

83%

1

lower data loss
probability

10-8

2288

1.6

10-6

1

EC is the best efficiency at equivalent durability compared toRAID6+1
1Hardware configuration Large Object Reference Architecture• *Other names and brands may be claimed as the property of others.
• Copyright ©2012 Intel Corporation.

Legal Disclaimer
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED,BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS
PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVERAND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING
LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANYPATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
• A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, inpersonal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICALAPPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, ANDTHE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSESAND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY,PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTELOR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OFITS PARTS.
• Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on theabsence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future
definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The
information here is subject to change without notice. Do not finalize a design with this information.
• The products described in this document may contain design defects or errors known as errata which may cause the product todeviate from published specifications. Current characterized errata are available on request.
• Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processorfamily, not across different processor families. Go to: http://www.intel.com/products/processor_number.
• Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
• Copies of documents which have an order number and are referenced in this document, or other Intel literature, may beobtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm• Intel, Xeon, Atom, Sponsors of Tomorrow and the Intel logo are trademarks of Intel Corporation in the United States and othercountries.
• *Other names and brands may be claimed as the property of others.
• Copyright ©2012 Intel Corporation.

• *Other names and brands may be claimed as the property of others.
• Copyright ©2012 Intel Corporation.