Spil Games Reduces Maintenceof High Availability with Galera Cluster
Spil Games is a social gaming company, with over 250 employees. It has grown in a short timefrom an internet start-up, to a global on-line gaming leader. It has now 49 gaming sites, serving
180 million users per month. This rapid growth of a business based very much on its database,and that has a large user-base that demands high-speed data service, presented some major
challenges to Art van Scheppingen, as Head of Database Engineering at Spil Games.

Inherent Difficulties andChallenges
As they began to finddifficulties
withtheadministration and maintenanceof their database system given
their particular needs, Art van
Scheppingen and his staff
looked to find a better
alternative. The main challenge
was to find a replacement for
their aging high availability
system that would work in bothold and new storage layers. Van
Scheppingen says, “The oldstorage layers were not much
more than direct database access from the applications.” They were using MMM with floating IP
addresses and one active master, along with an inactive master that could assume the role asmaster. This system writes only to the active master and performs read queries from the inactive
master and slaves.
The impact on their efforts had been mostly in the maintenance of these masters and slaves.
They needed to ensure that the inactive master and slaves were synchronized with the active
master. If one should fail, they would have to clone the master or another node to restore thefailed server. This was time consuming and reduced the availability within the cluster until thesituation was resolved. “Almost all monitoring tools to retain this type of HA are flawed: MMM,
MHa and Percona-PRM all have similar issues where they make too many decisions on theirown and try to repair the cluster by themselves (i.e., promote slaves to master),” says vanScheppingen.

Finding a Solution
To improve their situation, Spil Games looked for a solution that would provide them with thehigh availability they needed, consistent data between nodes, and easy maintenance. Theysought to abandon the method they had been using which performed separately reads andwrites, and looked to have a single interface to their database. They felt that MySQL replication
wasn't a good solution since it's single threaded within a single database. They wanted tomaintain a consistent state throughout the cluster. They found that all of these requirements andpreferences are met in Galera Cluster, with the assistance of Codership.

The main reason Spil Games decided to implement the Galera solution was due to ease of use.
“Once configured properly it is easy to add new nodes to a cluster. Once properly automatedusing Puppet, it becomes very easy also to create new clusters,” says van Scheppingen.
“Obviously, it took more than that to make the decision: we also investigated creating back-ups,load balancing (e.g., HA Proxy, MaxScale and Riverbed Stingray) and trying actually to push
the cluster to its limits.” In the end, they found Galera Cluster to be the best solution.
Spil Games now uses Galera in a three-server cluster, utilizing the Percona XtraDB Cluster
package. All of the servers are 16-core machines with at least 148GB of memory, and some use
flashcache. For load balancing, they use Riverbed Stingray load balancers since that was alreadypart of their infrastructure. For back-ups, they use Percona XtraBackup and stream the process
via the network to a centralized back-up solution (i.e., BackupPC).

Benefits
Now that Spil Games has deployed Galera Cluster, it has given them much more flexibility. “We
cut down on the number of components involved, reduced complexity and increased velocity,”says van Scheppingen. “We now can deploy a new cluster or add a new node a lot faster.” Thebenefits have been no data loss, no slave lag, 24/7 availability, and no complex fail-overs. All of
this has lead to no down-time or data loss, the retaining of customers, and the growth ofrevenues.
Van Scheppingen says, “Galera definitely lived up to our expectations: even though we had toinvest a lot of time and resources for the initital setup and tools, our maintenance is a lot easier
and well worth the investment for us.” For the future, Spil Games wants to expand the usage
within their sharded environment and to consolidate multiple, older active master/inactive masterdatabase pairs into Galera clusters. Van Scheppingen says, “We are very happy with Galera andare confident it will serve as our preferred replication/clustering solution for the upcomingyears.”

About Codership
Codership Ab provides high availability, no-data-loss and scalable data replication andclustering solutions for open source databases. Their flagship product is Galera Cluster™ forMySQL, a True Multimaster Cluster based on synchronous replication. Galera Cluster is an easyto-use, high-availability solution which provides high system uptime, no data loss and scalability
for future growth. For more information about Galera Cluster, please contact
info@codership.com