# Presentation: 175338

## Automated Tests in NICOS Nightly Control System

***Alexander Undrus***
**Brookhaven National Laboratory, Upton, NY 11973**

- Software testing is a difficult, time-consuming process that requires technical sophistication and proper planning. This is especially true for the large-scale software projects of High Energy Physics where constant modifications and enhancements are typical. The automated nightly testing is the important component of NICOS, NIghtly COntrol System, that manages the multi-platform nightly builds based on the recent versions of software packages. It facilitates collective work in collaborative environment and provides four benefits to developers: repeatability (tests can be executed more than once), accumulation (results are stored and reflected on NICOS web pages), feedback (automatic e-mail notifications about test failures), user friendly setup (configuration parameters can be encrypted in the body of test scripts). The modular structure of NICOS allows plugging in other validation and organization tools, such as QMTest and CppUNIT. NICOS classifies tests according to their granularity level and purpose. The low level structural tests reveal compilation problems, inconsistencies in package configuration, such as circular dependencies, and simple isolated bugs. The results for these three groups of tests are published for each package of  the software project. The integrated (or behavioral) tests find bugs at levels of users scenarios and NICOS generates the special web page with their results. The NICOS tool is currently used to coordinate the efforts of  more than 100 developers for the ATLAS project at CERN and included in the tool library of the LHC computing project.

**Abstract**

**CHEP 2004**

## NICOS - NIghtly COntrol System

- NICOS Control System [1] has a modular structure. Each module is responsible for a certain step (such as code checkout, release build, testing, error analysis) and independently described in the NICOS configuration file. This organization allows to plug in build, test, validation, and other external tools.
- NICOS currently manages the nightly releases of ATLAS software comprising about 1000 packages. It creates the nightly build framework that also uses:
-  ATLAS Tag Collector [2] – web interfaced database application. Developers are able to interactively select the tags from ATLAS CVS repository for the nightly releases
-  CMT [3] configuration management tool that defines the conventions for structuring software releases and describes the package properties, constituents, and dependencies

- CHEP 2004   -   Alexander Undrus   -   Automated Tests in NICOS Nightly Control System   -   page 2

## Test Levels and Tools in ATLAS Nightly Builds

**RELEASE**

**SYSTEM**

**PACKAGE**

**COMPONENT/UNIT**

**Verification of **
**compilation/linking**

**Unit Tests**

**Integration Tests**

**QA**** **
**(Configuration, **
**Coding Standards)**

- NICOS Control Tool provides a framework for tests of different manners and levels of focus. The tests are automatically applied to each nightly release.
-  The compilation and linking results are verified automatically and published for each package.
-  Quality Assurance tests verify  the descriptions of package properties (in CMT requirements files). The code checking tools are under development and will be added to the nightly tests in the future.
-  Unit tests check individual software components (such as C++ classes) or a collection of components (such as the packages of the ATLAS software release). A helper tool for unit testing of C++ programs, **CppUNIT **[4], is integrated in CMT environment of ATLAS releases. For testing individual packages the special *make* target, **make check**, is provided. It allows to run the test jobs focused on the package functionalities in *Athena, *ATLAS control framework.
-  Integration tests show that the major systems of the software release work well and communicate with other systems successfully. **QMTest** [5] tool is used for the ATLAS integration tests organization and validation. It supports regression tests that compare the output of the current test with previous (or known) values. The tests are arranged in suites. The suite usually includes the tests related to the specific system of a release. *QMTest* saves the test results for further inspection with the graphical or command-line interface.

- CHEP 2004   -   Alexander Undrus   -   Automated Tests in NICOS Nightly Control System   -   page 3

## Delivering Test Results to Developers

**Precise sw release info in header**

**Integrated **
**tests**
**statistics**

**Sorting options**

**Addresses**
**for e-mail**
**notifications**

**Build, QA and Unit tests results**

**Problems highlighted**

- NICOS web pages show the results of QA and unit tests for each package of a release. There are two levels of problem seriousness: *warning* and *error*. The results for integrated tests and suites are published on a separate web page.

**References**
- The NICOS home page is _[http://www.usatlas.bnl.gov/computing/software/nicos/](http://www.usatlas.bnl.gov/computing/software/nicos/)_
- S. Albrand “The Tag Collector – A Tool for Atlas Code Release Management”, CHEP04 contribution
- CMT home page is _[http://www.cmtsite.org](http://www.cmtsite.org/)_
- CppUNIT home page is _[http://sourceforge.net/projects/cppunit](http://sourceforge.net/projects/cppunit)_
- QMTest home page is _[http://www.codesourcery.com/](http://www.codesourcery.com/qmtest)__[qmtest](http://www.codesourcery.com/qmtest)_

- CHEP 2004   -   Alexander Undrus   -   Automated Tests in NICOS Nightly Control System   -   page 4