Halcyon: Unified HPC Center Operations

Under a shared campus cluster model, with many research groups investing distinct amounts and annual new hardware acquisitions, management of users and resources can become very complex. At Purdue, the Research Computing division set out in 2011 to design a cluster management solution to empower faculty to manage access to their own purchased resources. This system needed to also allow center staff to quickly provision resources, provide accurate accounting and tracking of faculty purchases over time, and provide a single location for data about the cluster program.<br> <br> This need was not unique to Purdue and, as more institutions enter into HPC services, the growing interest in similar systems is evidenced by new applications such as ColdFront. Perhaps most critical for the many “condo” style HPC centers, purchase of services can be tracked alongside other access information. This allows self-service ordering for internal accounting environments where purchases must go through organizational approvals, and all center purchase history may be correlated to allocations or other information. For Purdue, the user allocation/authorization alone was estimated as saving 6.7 hours of center staff time per week over the first 18 months.<br> <br> As operations at Purdue expanded, the internal portal took on many aspects of the operations of an HPC center beyond resource allocation and management. These aspects are seldom topics of discussion but often just as important to the success of an HPC center and include things such as customer relationship management, news, online presence, and documentation. Halcyon not only provides tools for these but integrations between them, allowing for such actions as emailing outage or maintenance news articles to users of a specific resource and auto-archiving of documentation when a resource is retired.<br> <br> While the original Purdue Research Computing portal served its purpose well over the years, making changes and adding features could be a cumbersome task. Reconstituting the portal features into a modular, extensible framework was an essential next step to allow for the growth and maintenance necessary. Re-architecture focused on:<br> <br> 1) Abstraction of environment assumptions and settings<br> 2) Restructure for modularity and easier extension<br> <br> To address (1), standard local language for many aspects , site content, and various hard-coded site settings were pulled into the internal database, and interfaces were created for editing these. Other environment site settings were incorporated through new configuration files stored in a central location.<br> <br> To address (2), the code was divided into modules—logical groupings based on the data being handled, tasks, and interfaces. Third-party services are integrated as plugins, allowing modules to ingest data from sources such as LDAP or REST APIs. Essential to interactions with external services, modules implement REST APIs conforming to the OpenAPI v3 specification.<br> <br> Halcyon provides a platform for integration between various self-service and admin-only elements such as resource allocation, access, communications, and knowledge. Halcyon allows for easier targeting and management of information critical to end users, and reduces time and effort on the part of center staff by integrating these missions.

Topic: Distributed and Parallel Computing Systems
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences