Characterization of load bearing metrological parameters in reptilian exuviae in comparison to precision-finished cylinder liner surfaces

The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous environment of commodity x86-64 processors and Linux. Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other applications on a dynamically shared pool of nodes. We present how we migrated our continuous integration system to schedule jobs on a relatively small Apache Mesos enabled cluster and how this resulted in better resource usage, higher peak performance and lower latency thanks to the dynamic scheduling capabilities of Mesos. 1. Current build infrastructure CMS Core Software group is responsible for the whole release engineering process for the CMS Offline Software, CMSSW. In particular this includes: • Release building and packaging, for both production releases and bi-daily integration builds for production and experimental branches alike. • Unit testing and reduced release validation for integration builds. • Automated Testing and Continuous Integration of updates as provided by various collaborators under the form of Github Pull Requests. • Running additional web based QA and documentation tools. • Deployment of nightly integration builds via CVMFS to the collaboration. CMS builds up to 21 releases per day on Scientific Linux 6 (SLC6) alone, including production releases, development ones and experimental branches. In the last month (April 7, 2015 – May 7, 2015) we had over 294 active pull requests tested (eventually multiple times) and integrated in 21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015) IOP Publishing Journal of Physics: Conference Series 664 (2015) 062013 doi:10.1088/1742-6596/664/6/062013 Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. Published under licence by IOP Publishing Ltd 1 FERMILAB-CONF-15-661-CMS ACCEPTED Operated by Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the United States Department of Energy. our release branches and integration. A total of over 8,000 pull requests have been processed by our Continuous Integration machinery since we started using Github in the middle of 2013. [1] To handle the automated QA and release workflows, the CMS build infrastructure currently consists of a cluster of 284 virtual cores and 540 GB of memory provisioned in the CERN OpenStack based cloud. [2] The QA procedures, the Continuous Integration are performed in an automatic manner via Jenkins [3], an OpenSource continuous integration system. Moreover the group is responsible for maintaining a number of web services, mostly serving build artifacts like installation RPMs, log files and ROOT files resulting from the tests. 2. Old, statically partitioned, build infrastructure architecture In the past in order to handle the load we could simply statically partition our OpenStack quota to have a good balance between the size of the build cluster and the size of the services. Moreover we provisioned build machines which were big enough for building a release with high level of parallelism, in order to reduce the build latency, but still small enough to avoid wasting to much CPU time when Jenkins scheduled non parallel jobs on them, wasting their potential. Moreover due to the fact that it’s not possible to cross compile SLC5 releases on SLC6, in the past some of the resources need to be assigned to SLC5 builds, which results in a net loss of efficiency since SLC5 builds are occasional but needed to be delivered with the same latency of SLC6 ones so we do not have much margin in overcommitting the SLC5 infrastructure. While the static partitioning of the cluster has been sufficient for a long period, we wish to improve both the latency for critical builds (i.e. development builds or tests on pull requests for development builds) and improve the overall cluster utilization, with the aim of reducing the number of resources needed. 3. Apache Mesos: a dynamic approach to cluster partitioning The latency and resource utilization problems we are facing are actually a common issue in the industry, where static partitioning of resources is the typical cause of resource overcommitting and the following low cluster utilization. Big names in the industry (e.g. Google [4]) have long solved the problem with proprietary solutions which provide more dynamic cluster scheduling, however recently a number of opensource projects have emerged trying to provide the same kind of functionality. In particular we decided to evaluate a resource manager called Apache Mesos [5] (just Mesos from now on). Mesos originated from the research done at UC Berkeley [6] which has then been brought to production by Twitter. As the project evolved it got incubated as part of the Apache Foundation Incubator, successfully graduating from it in 2013 [7], and it’s therefore now distributed as flagship project under the Apache License. It is commonly used by other big internet companies like Apple, eBay, Vimeo, and universities like UCSF and UC Berkeley. Mesos cluster architecture consists of two sets of nodes, masters which coordinate the work, and slaves which execute it. In particular Mesos is a so called two-level scheduler: the slave nodes offer their available resources to masters which are then responsible for handling them to the so called frameworks standalone software components which are responsible for the application specific scheduling policy. When proposed a resource offer, a framework can decide whether to accept it or not, and in case it does it can specify which and how many tasks it would like to create for the given resource offer. This allows for more fine tuned scheduling of resources and for application aware scheduling and has been demonstrated in very large scale (thousands of nodes) setups. On the back-end, Mesos masters uses Apache ZooKeeper (just ZooKeeper from now on) [8] to provide a fault tolerant, distributed, setup when multiple masters are used. 21st International Conference on Computing in High Energy and Nuclear Physics (CHEP2015) IOP Publishing Journal of Physics: Conference Series 664 (2015) 062013 doi:10.1088/1742-6596/664/6/062013