Beowulf cluster

Beowulf cluster
A Beowulf cluster is a computer cluster of what are normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed which allow processing to be shared among them. The result is a high-performance parallel computing cluster from inexpensive personal computer hardware.
The name Beowulf originally referred to a specific computer built in 1994 by Thomas Sterling and Donald Becker at NASA. The name "Beowulf" comes from the Old English epic poem of the same name.
No particular piece of software defines a cluster as a Beowulf. Typically only free and open source software is used, both to save cost and to allow customization. Most Beowulf clusters run a Unix-like operating system, such as BSD, Linux, or Solaris. Commonly used parallel processing libraries include Message Passing Interface (MPI) and Parallel Virtual Machine (PVM). Both of these permit the programmer to divide a task among a group of networked computers, and collect the results of processing. Examples of MPI software include Open MPI or MPICH. There are additional MPI implementations available.
Beowulf systems operate worldwide, chiefly in support of scientific computing. Since 2017, every system on the Top500 list of the world's fastest supercomputers has used Beowulf software methods and a Linux operating system. At this level, however, most are by no means just assemblages of commodity hardware; custom design work is often required for the nodes (often blade servers), the networking and the cooling systems.
Development
A description of the Beowulf cluster, from the original "how-to", which was published by Jacek Radajewski and Douglas Eadline under the Linux Documentation Project in 1998:
Operating systems
a number of Linux distributions, and at least one BSD, are designed for building Beowulf clusters. These include:
MOSIX, geared toward computationally intensive, IO-low applications
Rocks Cluster Distribution, latest 2017
DragonFly BSD OS, latest 2022
Quantian OS latest 2006, a live DVD with scientific applications, remastered from Knoppix
Kentucky Linux Athlon Testbed, physical installation at University of Kentucky
The following are no longer maintained:
Kerrighed (EOL: 2013)
OpenMosix (EOL: 2008), forked from MOSIX
ClusterKnoppix OS, forked from Knoppix OS, forked from OpenMosix
PelicanHPC OS latest 2016, based on Debian Live
A cluster can be set up by using Knoppix bootable CDs in combination with OpenMosix. The computers will automatically link together, without need for complex configurations, to form a Beowulf cluster using all CPUs and RAM in the cluster. A Beowulf cluster is scalable to a nearly unlimited number of computers, limited only by the overhead of the network.
Provisioning of operating systems and other software for a Beowulf Cluster can be automated using software, such as Open Source Cluster Application Resources. OSCAR installs on top of a standard installation of a supported Linux distribution on a cluster's head node.
See also
Aiyara cluster
HTCondor
Grid computing
Kentucky Linux Athlon Testbed
Maui Cluster Scheduler
Open Source Cluster Application Resources
Stone Soupercomputer
Oracle Grid Engine
References
Bibliography
Beowulf Cluster Computing With Windows by Thomas Lawrence Sterling 2001 MIT Press
Beowulf Cluster Computing With Linux by Thomas Lawrence Sterling 2001 MIT Press
External links
The Beowulf Cluster Site
A detailed description of building a Beowulf cluster
Cluster computing
Parallel computing
Job scheduling
NASA spin-off technologies
Unix software