--------------------------------------- -------------------------------- ---------------------------
  ![](media/image1.emf){width="0.875in"   **Fermilab**                     
  height="0.8194444444444444in"}                                           

  --------------------------------------- -------------------------------- ---------------------------

**MEMORANDUM OF UNDERSTANDING**

**Between the**

**MINOS Experiment and the Fermilab Computing Division**

June 22 2005

Version 1.1

[INTRODUCTION 3](#introduction)

[I. PERSONNEL AND INSTITUTIONS 3](#i.-personnel-and-institutions)

[II. Fermilab Computing Division 3](#ii.-fermilab-computing-division)

[III. SPECIAL CONSIDERATIONS 4](#iii.-special-considerations)

[IV. SIGNATURES 5](#iv.-signatures)

[V. NETWORKING SUPPORT 6](#v.-networking-support)

[VI. OFFLINE COMPUTING 8](#vi.-offline-computing)

[OVERVIEW 8](#overview)

[MINOS 5 YEAR RUN PLAN 8](#minos-5-year-run-plan)

[EVENT RECONSTRUCTION AND STORAGE 10](#event-reconstruction-and-storage)

[MONTE CARLO GENERATION AND STORAGE
12](#monte-carlo-generation-and-storage)

[DATA HANDLING AND STORAGE 15](#data-handling-and-storage)

[OFFLINE ANALYSIS 16](#offline-analysis)

[DATABASE 18](#database)

[SUMMARY 19](#summary)

[SUPPORT REQUIREMENTS 20](#support-requirements)

#  {#section .unnumbered}

# 

# INTRODUCTION

This is a memorandum of understanding between the Fermi National
Accelerator Laboratory Computing Division (CD) and the experimenters of
MINOS (E-875). The memorandum is intended solely for the purpose of
providing a budget estimate and guidance for effort allocated by
Fermilab, the funding agencies and the participating institutions. It
reflects an arrangement that currently is satisfactory to the parties;
however, it is recognized and anticipated that changing circumstances of
the evolving research program will necessitate revisions. The parties
agree to negotiate amendments to this memorandum that will reflect such
required adjustments.

# I. PERSONNEL AND INSTITUTIONS

Co-spokesperson: S. Wojcicki

Co-spokesperson: D. Michael

Deputy Spokesperson: D. Ayres

Relevant MINOS System Managers:

Offline Computing J. Urheim

# II. Fermilab Computing Division 

1.  The Computing Division liaison is E. Buckley-Geer.

2.  The off-line analysis plan in Section VI contains the experiment\'s
    present understanding of its offline needs for data storage, event
    reconstruction, data analysis and Monte Carlo generation. The
    Computing Division cannot guarantee, at this time, that these
    resources can be made available. The Computing Division, guided by
    priorities set by laboratory management, will attempt to allocate
    the requested resources. The present request and amendments will be
    used in attempting to plan the laboratory\'s computing acquisition
    strategies.

3.  Cost Accounting FY05 and beyond: The Fermilab Computing will
    maintain chargeable tasks specific to the MINOS experiment, and all
    MINOS associated expenditures will be charged to those tasks.

# III. SPECIAL CONSIDERATIONS

1.  For the purpose of estimating budgets, specific products and vendors
    may be mentioned within this memorandum. At the time of purchasing,
    the Fermilab procurement policies shall apply. This may result in
    the purchase of different products and/or from different vendors.

# IV. SIGNATURES

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

V. White, Head of Fermilab Computing Division

# 

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

S. Wojcicki, MINOS Co-spokesperson

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

D. Michael, MINOS Co-spokesperson

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

J. Urheim, MINOS Offline Computing Manager

# V. NETWORKING SUPPORT 

The Far detector LAN at Soudan and the Near detector LAN at Fermilab
have been configured and installed by the Fermilab network group.

The strategy for the Soudan LAN support is to incorporate the operation
and management into the Fermilab campus network support effort. The
Fermilab policy that defines the campus network as a restricted central
service (<http://www.fnal.gov/cd/main/cpolicy.pdf>) will be extended to
include the local network at Soudan. From a practical perspective this
will mean:

1.  The topology of the Soudan network will be specified by the Fermilab
    network group and modifications or changes made only in consultation
    with or at the direction of that group.

2.  All active network devices (switches, routers, hubs with manageable
    components) will be procured and installed under the direction of
    the Fermilab network group and will be managed remotely by that
    group.

3.  Basic network services (DNS, DHCP) will be provided on servers that
    will be supplied, installed and remotely managed by the Fermilab
    network group.

4.  Any maintenance on the LAN and its components should be scheduled in
    consultation with the experiment.

5.  During data-taking with beam there should be 24/7 coverage for
    resolution of problems at both Fermilab and Soudan.

6.  A liaison from the Fermilab network group should be provided at all
    times.

7.  Sufficient spares should be kept at Soudan so that delays are not
    incurred in resolving problems due to unavailability of spares.

> The remote management of the Soudan LAN is predicated on having a
> simple, easily manageable architecture. This necessitates centralized
> network connections and a minimum number of active network devices.
> Un-intelligent microhubs may be used to provide extra network
> connections in situations where all the local network jacks are in
> use. All active network devices, including un-intelligent microhubs,
> will be of a type, make and model specified or approved by the
> Fermilab network group.
>
> [Network Configuration]{.underline}

At Soudan the network domain is minos-soudan.org (198.124). There are
four pools of addresses

  -------------- --------------------------------------------------------
  DHCP pool      191.124.212.0 with addresses from 1-199 and 202-253

  SERVER LAN     198-124-213.0 with addresses from 1-29

  DCS LAN        198.124.213.64 with addresses from 65-125

  DAQ LAN        198.124.213.128 with addresses from 129-253
  -------------- --------------------------------------------------------

The Near Detector LAN will be subnet 131.225.192. There will be three
pools of addresses

  -------------- --------------------------------------------------------
  DHCP pool      131.225.192.192 with addresses from 193-221

  DCS LAN        131.225.192.0 with addresses from 1-125

  DAQ LAN        131.225.192.128 with addresses from 129-189
  -------------- --------------------------------------------------------

The DCS and DAQ LANs permit access to each other and reflexive replies
to traffic that originated from within those LANs. Login access to these
LANs is allowed through a fully strengthened \"gateway\" connected to
the SERVER LAN at Soudan and to the general site network at Fermilab. To
support the kerberized access there is a satellite Key Distribution
Server (KDC) installed at Soudan. This machine is administered remotely
by members of the Fermilab Computer Security team.

The LAN at the near detector has fiber connected directly to the
front-end electronics racks with media converters to avoid any
possibility of ground loops.

We do not expect to have any special requirements for networking in FCC
beyond what is generally available.

[Network Time Protocol (NTP) server]{.underline}

MINOS will run their own NTP server at both the Far and Near detector.

The strategy is to synchronize computers locally using the GPS system at
each site. However, for monitoring purposes, we will exchange
information with both a trusted timeserver on the web and to the other
MINOS detector (near from far and far from near). Our modified NTP
servers will not use this information to set the clock, but will make
the offsets available to be written to the data stream. This should
allow us, if something unexpected happens in the data to be able to
figure out what happened. It is therefore vital in that we are able to
send NTP packets in both directions between Fermilab and Soudan and also
that we can send packets to external time servers (but we will not be
accepting unsolicited requests for time to be served, and will see any
attempts).

# VI. OFFLINE COMPUTING

## OVERVIEW

This section deals with the offline computing needs for the MINOS
experiment. The offline needs can be broken into the following areas:

-   Data Handling and Storage

```{=html}
<!-- -->
```
-   Offline Data Processing

-   Monte Carlo Generation

-   Offline Analysis

-   Software

-   Support requirements

This document will address the resources required in each of the
preceding areas.

We would like as much as is reasonable to integrate our computing
resources into the existing centrally supported architecture to reduce
the support load both from MINOS and from CD. We expect to use
Sequential Access with Metadata (SAM) for data handling which was
developed for the Run II experiments. We will continue to use the
central Linux machines (currently called FNALU) for interactive
development and will supplement the interactive capability with a small
farm of Linux nodes. We will continue to use the existing Linux batch
nodes and would like to add additional nodes for MINOS use. We will
continue to use AFS disk for user data. We expect that our FARM needs
would be met by augmenting the existing general purpose farm. The
overall needs for hardware are summarized in Table 1. The CPU needs for
data processing and Monte Carlo include the requirements for
reprocessing which are somewhat uncertain. We assume that at least half
the Monte Carlo production needs will be met from offsite resources.

  ------------------------ ------ ------ ------- ------- -------- ---------
  Year                     2004   2005   2006    2007    2008     2009

  Tape (TB)                6      39     60      35      41       162

  Disk (TB)                2.4    13     20      17      21       56

  CPU, Reco (GHz)          55     175    427     110     7        1013

  CPU, MC (GHz)            132    915    963     560     63       6630

  CPU MC (GHz) at FNAL     66     458    482     280     32       3315

  CPU, analysis (GHz)      80     154    170     590     200      230
  ------------------------ ------ ------ ------- ------- -------- ---------

Table 1 Summary of MINOS hardware needs for 6 years of data taking.

## MINOS 5 YEAR RUN PLAN

The experiment has presented a request to the laboratory management for
the number of protons on target per year (POT/year). This request is
summarized in Table 2.

  ------------------- ------------------- -------------------------------
  Year                POT x 10^20^        Protons per pulse x 10^13^

  2005                2.5                 2.5

  2006                3                   3

  2007                3.5                 3.5

  2008                4                   4

  2009                5                   5
  ------------------- ------------------- -------------------------------

Table 2 Requested number of protons for the MINOS run plan

The rate and raw data volumes for the Far detector are shown in Table 3
and for the Near Detector in Table 4. The beam neutrino rate in the far
detector has been neglected, as it is a small fraction of the combined
cosmic ray and noise rates. The rate from the far detector is dominated
by 0.55 Hz of cosmic ray interactions, which will be used both for
calibration and for cosmic ray and atmospheric neutrino physics studies.
The far detector also has about 6.5 Hz of noise triggers, which are
small events and contribute to the raw data size but are eliminated
after reconstruction. There are also pedestal runs, light injection etc
which are listed in the "Other" category. The neutrino interaction rate
for the near detector will vary depending on the number of protons on
target as will the spill size. The spill size is quoted for the lowest
number of protons on target and are scaled for the higher intensities.
The near detector DAQ system is capable of recording the full 250 Hz of
cosmic rays seen by the near detector but it is expected that we will
record only a fraction of these calibration purposes, we have assumed 10
Hz, and this is reflected in the numbers in Table 4. The far detector
assumes 310^7^ seconds in one year (cosmic rays are always there) and
the near detector assumes an effective year of 210^7^ seconds for beam
and 310^7^ seconds for the cosmic rays. For simplicity we assume that
1Kbyte 1000 bytes.

+-----------+---------------+---------------+-------------+------------+
| Sample    | Rate/second   | Events/year   | Raw Event   | Data       |
|           |               |               |             |            |
|           | (Hz)          |               | Size        | Volume     |
|           |               |               | (Kbytes)    |            |
|           |               |               |             | /year (GB) |
+-----------+---------------+---------------+-------------+------------+
| Cosmic    | 0.55          | 1.6510^7^     | 1.1         | 18         |
|           |               |               |             |            |
| ray       |               |               |             |            |
+-----------+---------------+---------------+-------------+------------+
| Noise     | 6.5           | 1.95 10^8^    | 0.2         | 39         |
+-----------+---------------+---------------+-------------+------------+
| Other     |               |               |             | 312        |
+-----------+---------------+---------------+-------------+------------+
| Total     |               |               |             | 369        |
+-----------+---------------+---------------+-------------+------------+

Table 3 Event rates and raw data volumes for the Far detector

+-----------------------+-----------+------+------+------+------+------+
|                       |           | Raw  |      |      |      |      |
|                       |           | Data |      |      |      |      |
|                       |           | Vo   |      |      |      |      |
|                       |           | lume |      |      |      |      |
|                       |           | per  |      |      |      |      |
|                       |           | year |      |      |      |      |
|                       |           | (GB) |      |      |      |      |
+-----------------------+-----------+------+------+------+------+------+
| Sample                | Spill     | 2005 | 2006 | 2007 | 2008 | 2009 |
|                       | Size      |      |      |      |      |      |
|                       |           |      |      |      |      |      |
|                       | (Kbytes)  |      |      |      |      |      |
+-----------------------+-----------+------+------+------+------+------+
| Beam                  | 10        | 100  | 120  | 140  | 160  | 200  |
+-----------------------+-----------+------+------+------+------+------+
| Cosmic Rays           | 0.6       | 180  | 180  | 180  | 180  | 180  |
+-----------------------+-----------+------+------+------+------+------+
| Total                 |           | 280  | 300  | 320  | 340  | 380  |
+-----------------------+-----------+------+------+------+------+------+

Table 4 Raw Data volumes for the Near detector

## EVENT RECONSTRUCTION AND STORAGE

The event reconstruction for both detectors will be done at Fermilab.
The Far detector reconstruction of cosmic rays and atmospheric neutrino
events has been running on the FNAL general purpose farm for about 2
years. A summary of the processing needs is given in Table 5 for steady
state, which will keep up with the data taking, and in Table 6 for
reprocessing. These numbers are based on the performance of the existing
MINOS C++ reconstruction code. The Far detector numbers are taken from
real data, the Near detector from Monte Carlo. The processing time per
event or spill will be given in GHz-seconds per event and the CPU
requirements will be given in GHz. The Near detector processing times
are increased to account for the higher beam intensity over time. We
have assumed that in the years 2004 and 2005 we will do one complete
reprocessing pass of the Far detector data per year that will be
completed in 3 months. For the Near detector we assume one reprocessing
per year in 2005 and 2006, again taking 3 months. In 2009 we assume one
pass of all the data taking 6 months. The reprocessing numbers include
processing for all the data taken up until that time. We assume a farm
efficiency of 70%.

  ----------------- ---------------- ------ ------ ------ ------ ------ ------
                                     GHz                                
                                     per                                
                                     year                               

                    Execution time   2004   2005   2006   2007   2008   2009

  Beam data (Near)  70 GHz-sec/spill        33     40     47     53     70

  Cosmic Rays       4 GHz-sec/event         60     60     60     60     60
  (Near)                                                                

  Cosmic Rays (Far) 6.3              5      5      5      5      5      5
                    GHz-sec/event                                       

  Total                              5      98     105    112    118    135
  ----------------- ---------------- ------ ------ ------ ------ ------ ------

Table 5 Steady state event reconstruction needs for the Near and Far
detectors

+-------------------------+---------+---------+---------+------------+
|                         | GHz per |         |         |            |
|                         | reconst |         |         |            |
|                         | ruction |         |         |            |
|                         | pass    |         |         |            |
+-------------------------+---------+---------+---------+------------+
|                         | 2004    | 2005    | 2006    | 2009       |
+-------------------------+---------+---------+---------+------------+
| Beam Data (Near)        |         | 70      | 210     | 400        |
+-------------------------+---------+---------+---------+------------+
| Cosmic Rays             |         | 340     | 340     | 510        |
|                         |         |         |         |            |
| (Near)                  |         |         |         |            |
+-------------------------+---------+---------+---------+------------+
| Cosmic Rays             | 50      | 70      |         | 80         |
|                         |         |         |         |            |
| (Far)                   |         |         |         |            |
+-------------------------+---------+---------+---------+------------+
| Total                   | 50      | 480     | 550     | 990        |
+-------------------------+---------+---------+---------+------------+

Table 6 Reprocessing needs for Near and Far detectors

The raw data is highly compressed but it expands after event
reconstruction when we add the de-multiplexed hits, tracking information
and calibration. The current expansion rate is a factor of 30. We are
studying this to see if it can be reduced but we have used this number
in the planning that follows. In Tables 7 and 8 we show the expected
reconstructed data volumes for the near and far detectors. We currently
write a file of Candidates, an Ntuple and a compressed Ntuple. These are
included in the calculations. We have used the beam event size for Near
detector cosmic ray events.

+-----------------+----------+-------+-----+-----+-----+-----+--------+
|                 |          |       | R   |     |     |     |        |
|                 |          |       | eco |     |     |     |        |
|                 |          |       | nst |     |     |     |        |
|                 |          |       | ruc |     |     |     |        |
|                 |          |       | ted |     |     |     |        |
|                 |          |       | D   |     |     |     |        |
|                 |          |       | ata |     |     |     |        |
|                 |          |       | Vol |     |     |     |        |
|                 |          |       | ume |     |     |     |        |
|                 |          |       | per |     |     |     |        |
|                 |          |       | y   |     |     |     |        |
|                 |          |       | ear |     |     |     |        |
|                 |          |       | (   |     |     |     |        |
|                 |          |       | GB) |     |     |     |        |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
| Sample          | Data     | Spill | 2   | 2   | 2   | 2   | 2009   |
|                 | type     | Size  | 005 | 006 | 007 | 008 |        |
|                 |          |       |     |     |     |     |        |
|                 |          | (Kb   |     |     |     |     |        |
|                 |          | ytes) |     |     |     |     |        |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
| Beam Data       | C        | 236   | 2   | 2   | 3   | 3   | 4720   |
|                 | andidate |       | 400 | 800 | 300 | 800 |        |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
|                 | Ntuple   | 27    | 270 | 320 | 380 | 430 | 540    |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
|                 | Comp.    | 10    | 100 | 120 | 140 | 160 | 200    |
|                 | Ntuple   |       |     |     |     |     |        |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
| Cosmic Rays     | C        | 20    | 6   | 6   | 6   | 6   | 6000   |
|                 | andidate |       | 000 | 000 | 000 | 000 |        |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
|                 | Ntuple   | 2     | 600 | 600 | 600 | 600 | 600    |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
|                 | Comp.    | 0.5   | 150 | 150 | 150 | 150 | 150    |
|                 | Ntuple   |       |     |     |     |     |        |
+-----------------+----------+-------+-----+-----+-----+-----+--------+
| Total           |          |       | 9   | 9   | 10  | 11  | 12210  |
|                 |          |       | 520 | 900 | 570 | 140 |        |
+-----------------+----------+-------+-----+-----+-----+-----+--------+

Table 7 Reconstructed data volumes for the Near detector

  ----------------- ----------------- ----------------- -----------------
  Sample            Data Type         Event Size        Reconstructed
                                      (Kbytes)          Data volume per
                                                        year (GB)

  Cosmic Rays       Candidate         30                500

                    Ntuple            6                 100

                    Comp. Ntuple      1                 20

  Total                                                 620
  ----------------- ----------------- ----------------- -----------------

Table 8 Reconstructed data volumes for the Far detector

In Tables 9 and 10 we show the additional data volumes generated by
reprocessing the data.

+-----------------------+-------------+---------+----------+----------+
|                       |             | Repr    |          |          |
|                       |             | ocessed |          |          |
|                       |             | Data    |          |          |
|                       |             | Volume  |          |          |
|                       |             |         |          |          |
|                       |             | per     |          |          |
|                       |             | year    |          |          |
|                       |             | (GB)    |          |          |
+-----------------------+-------------+---------+----------+----------+
| Sample                |             | 2005    | 2006     | 2009     |
+-----------------------+-------------+---------+----------+----------+
| Beam data             | Candidate   | 1180    | 3800     | 14600    |
+-----------------------+-------------+---------+----------+----------+
|                       | Ntuple      | 140     | 430      | 1700     |
+-----------------------+-------------+---------+----------+----------+
|                       | Comp.       | 50      | 160      | 620      |
|                       | Ntuple      |         |          |          |
+-----------------------+-------------+---------+----------+----------+
| Cosmic Rays           | Candidate   | 3000    | 9000     | 27000    |
+-----------------------+-------------+---------+----------+----------+
|                       | Ntuple      | 300     | 900      | 2700     |
+-----------------------+-------------+---------+----------+----------+
|                       | Comp.       | 80      | 230      | 680      |
|                       | Ntuple      |         |          |          |
+-----------------------+-------------+---------+----------+----------+

Table 9 Reprocessed data volumes for the Near detector

+-----------------------+-------------+---------+----------+----------+
|                       |             | Repr    |          |          |
|                       |             | ocessed |          |          |
|                       |             | Data    |          |          |
|                       |             | Volume  |          |          |
|                       |             |         |          |          |
|                       |             | per     |          |          |
|                       |             | year    |          |          |
|                       |             | (GB)    |          |          |
+-----------------------+-------------+---------+----------+----------+
| Sample                |             | 2004    | 2005     | 2009     |
+-----------------------+-------------+---------+----------+----------+
| Cosmic Rays           | Candidate   | 1240    | 1730     | 3710     |
+-----------------------+-------------+---------+----------+----------+
|                       | Ntuple      | 250     | 350      | 740      |
+-----------------------+-------------+---------+----------+----------+
|                       | Comp.       | 40      | 60       | 120      |
|                       | Ntuple      |         |          |          |
+-----------------------+-------------+---------+----------+----------+

Table 10 Reprocessed data volumes for the Far detector

## MONTE CARLO GENERATION AND STORAGE

There are two types of Monte Carlo required for MINOS, simulation of
neutrino interactions in the detector for oscillation
measurements/conventional neutrino physics and simulation of the
neutrino beam to understand features of the beam such as beam profiles,
flux etc. In both cases the requirements are not precisely known so the
numbers here are based on assumptions. We give the total requirements
here but we assume that at least 50% of the Monte Carlo could be
generated by collaborating institutions and transferred to Fermilab for
storage.

[Physics Monte Carlo]{.underline}

For studies of cosmic ray and atmospheric neutrino events in the Far
detector we assume a Monte Carlo sample equal to the data. For the Near
detector we have made a similar assumption. The event size is larger due
to storage of the "truth" information" for the event. The needs per year
are summarized in Table 11.

  --------------------------------- ------ ------ ----- ----- ------ ----- ------ ----- ------ ------ -----
                                    Year                                                              

                                    2004   2005               2006         2007         2008   2009   

  **Far detector Cosmic Rays**                                                                        

  Events (10^7^)                    1.65   1.65               1.65         1.65         1.65   1.65   

  Raw Data (TB) (50 Kbytes/event)   0.8    0.8                0.8          0.8          0.8    0.8    

  Candidates (TB) (90 Kbytes/event) 1.5    1.5                1.5          1.5          1.5    1.5    

  Ntuple (TB) (13 Kbytes/event)     0.2    0.2                0.2          0.2          0.2    0.2    

  Comp. Ntuple (TB) (3              0.05   0.05               0.05         0.05         0.05   0.05   
  Kbytes/event)                                                                                       

  CPU time/event (GHz-sec)          16     16                 16           16           16     16     

  CPU for 2 months (GHz)            70     70                 70           70           70     70     

  **Far detector Beam**                                                                               

  Events (10^6^)                    2             2     2            2            2                   2

  Raw Data (GB) (80 Kbytes/event)   160           160   160          160          160                 160

  Candidates (GB) (160              320           320   320          320          320                 320
  Kbytes/event)                                                                                       

  Ntuple (GB) (6 Kbytes/event)      10            10    10           10           10                  10

  Comp. Ntuple (GB) (1 Kbyte/event) 2             2     2            2            2                   2

  CPU time/event (GHz-sec)          38            38    38           38           38                  38

  CPU for 1 month (GHz)             40            40    40           40           40                  40

  **Near detector Beam**                                                                              

  Spills (10^7^)                           1                  1            1            1      1      

  Raw Data (TB) (450 Kbytes/spill)         4.5                5.4          6.3          7.2    9      

  Candidates (TB) (780                     8                  9            11           13     16     
  Kbytes/spill)                                                                                       

  Ntuple (GB) (36 Kbytes/spill)            360                430          500          580    720    

  Comp. Ntuple (GB) (10                    100                120          140          160    200    
  Kbytes/spill)                                                                                       

  CPU time/spill (GHz-sec)                 340                340          340          340    340    

  CPU for 6 months (GHz)                   320                380          440          500    630    
  --------------------------------- ------ ------ ----- ----- ------ ----- ------ ----- ------ ------ -----

Table 11 Physics Monte Carlo needs per year

This data will also need to be stored in the STKEN tape library.

We expect to reprocess the Monte Carlo data with the same frequency as
the data. These requirements are shown in Table 12 for CPU and Table 13
for data volumes.

  -------------------------- ---------------- ---------- ---------- ------------
                             **GHz per                              
                             reconstruction                         
                             pass**                                 

                             2004             2005       2006       2009

  Near Beam MC                                470        1500       5800

  Far Cosmic MC              12               90                    410

  Far Beam MC                7                50                    230

  Total                      19               610        1500       6440
  -------------------------- ---------------- ---------- ---------- ------------

Table 12 CPU for reprocessing for the Far and Near Monte Carlo

+-------------------+-----------+-------+--------+--------+---------+
|                   |           | R     |        |        |         |
|                   |           | eproc |        |        |         |
|                   |           | essed |        |        |         |
|                   |           | Data  |        |        |         |
|                   |           | V     |        |        |         |
|                   |           | olume |        |        |         |
|                   |           |       |        |        |         |
|                   |           | per   |        |        |         |
|                   |           | year  |        |        |         |
|                   |           | (GB)  |        |        |         |
+-------------------+-----------+-------+--------+--------+---------+
| Sample            |           | 2004  | 2005   | 2006   | 2009    |
+-------------------+-----------+-------+--------+--------+---------+
| Far Cosmic MC     | Candidate | 250   | 1730   |        | 8170    |
+-------------------+-----------+-------+--------+--------+---------+
|                   | Ntuple    | 40    | 250    |        | 1180    |
+-------------------+-----------+-------+--------+--------+---------+
|                   | Comp.     | 8     | 60     |        | 340     |
|                   | Ntuple    |       |        |        |         |
+-------------------+-----------+-------+--------+--------+---------+
| Far Beam MC       | Candidate | 30    | 190    |        | 880     |
+-------------------+-----------+-------+--------+--------+---------+
|                   | Ntuple    | 50    | 370    |        | 1760    |
+-------------------+-----------+-------+--------+--------+---------+
|                   | Comp.     | 2     | 10     |        | 70      |
|                   | Ntuple    |       |        |        |         |
+-------------------+-----------+-------+--------+--------+---------+
| Near Beam MC      | Candidate |       | 3900   | 12500  | 48400   |
+-------------------+-----------+-------+--------+--------+---------+
|                   | Ntuple    |       | 180    | 580    | 2200    |
+-------------------+-----------+-------+--------+--------+---------+
|                   | Comp.     |       | 50     | 160    | 620     |
|                   | Ntuple    |       |        |        |         |
+-------------------+-----------+-------+--------+--------+---------+

Table 13 Reprocessed data volumes for the Far and Near Monte Carlo

[Beam Monte Carlo]{.underline}

Currently a single run takes about 2.410^5^ GHz-seconds to obtain
sufficient statistics. The data volume produced per run is about 240
Mbytes. It is expected that a factor of 10 times longer runs will be
needed for the physics analysis and that a few hundred of these runs
will be required. We currently assume that the runs are generated over
the course of a year. The needs are summarized in Table 14.

  ----------------------------------- -----------------------------------
  GHz-sec/run                         2.410^6^

  Runs/year                           200

  GHz/year                            15

  Storage/year                        480 GB
  ----------------------------------- -----------------------------------

Table 14 Beam Monte Carlo needs per year

## DATA HANDLING AND STORAGE

In calculating the amount of storage required we assume that recycling
of storage occupied by older reprocessing passes will be carried out. We
assume that at any one time there are two processing passes being
stored.

Table 15 summarizes amount of storage required per year including
reprocessing and allowing for recycling.

  ------------------- ------------ ---------- ------- ------- ------- ------- ------
                                   Storage                                    
                                   required                                   
                                   per year                                   
                                   (TB)                                       

                                   2004       2005    2006    2007    2008    2009

  Far detector        Candidate    1.7        2.7     1.5     0.5     0.5     3.2
  Cosmics                                                                     

                      Ntuple       0.3        0.4     0.3     0.3     0.3     0.6

                      Comp. Ntuple 0.06       0.07    0.05    0.05    0.05    0.1

  Near detector       Candidate               9       15      6       6       33
  Cosmics                                                                     

                      Ntuple                  0.9     1.5     0.6     0.6     3.3

                      Comp. Ntuple            0.2     0.4     0.2     0.2     0.8

  Near detector beam  Candidate               3.5     6.6     3.3     3.8     19

                      Ntuple                  0.4     0.8     0.4     0.4     2.2

                      Comp. Ntuple            0.2     0.3     0.2     0.2     1.0

  Far detector Cosmic Candidate    2.2        3.7     1.5     1.5     1.5     9.7
  MC                                                                          

                      Ntuple       0.3        0.5     0.2     0.2     0.2     1.4

                      Comp. Ntuple 0.07       0.1     0.05    0.05    0.05    0.3

  Far detector beam   Candidate               0.5     0.8     0.3     0.3     1.8
  MC                                                                          

                      Ntuple                  0.02    0.03    0.01    0.01    0.07

                      Comp. Ntuple            0.003   0.005   0.002   0.002   0.01

  Near detector beam  Candidate               12      22      11      13      64
  MC                                                                          

                      Ntuple                  0.5     1.0     0.5     0.6     3

                      Comp. Ntuple            0.2     0.3     0.2     0.2     0.4

  Total                            5          34      52      25      28      144
  ------------------- ------------ ---------- ------- ------- ------- ------- ------

Table 15 Storage required per year including reprocessing and allowing
for recycling.

All data will be stored in the STKEN tape robot. All the Ntuple-level
data will be stored permanently on disk. We plan for DCache disk to
stage about 20% of the remaining data at any one time.

We will record data on disk locally at the detectors and transfer it
over the network to Fermilab/FCC (Feynman Computing Center) into DCache
for archiving to tape. We plan to make several copies of the raw data as
the volume is small and tapes are large, meaning a tape failure could
result in significant data loss.

## OFFLINE ANALYSIS

Analysis of the Far detector data is currently being done at Fermilab as
well as at other MINOS collaborating institutions. This will expand to
include the Near detector. We currently have limited analysis experience
so we have constructed an analysis model which will have to be revised
once we get data. We assume that there will be of order 30 users doing
analysis on a central MINOS facility located at FNAL. Most analyses will
proceed from the Ntuples. We assume that most users are using the
Ntuples produced on the farm but that a few, of order five might need to
produce their own Ntuples from the Candidate files.

######## Ntuple analysis

We assume 30 users and each makes 6 passes per year through the Ntuples
to create a custom ntuple. It typically takes 0.0004 GHz-seconds/event
to produce an Ntuple record using the standard production Far detector
Ntuples as input. We assume that a user should be able to produce a
custom Ntuple from the Far detector data and Monte Carlo in 24 hours. We
assume that one pass always includes all the data from previous years.
For the Near detector we assume that users take 1 week to create a
compressed Ntuple from the data and the Monte Carlo. It takes about
0.003 GHz-sec/spill to create an Ntuple from the Near detector
production Ntuples.

  ---------- --------- ---------- ---------- ---------- ---------- ----------
             CPU in                                                
             GHz                                                   

  Year       2004      2005       2006       2007       2008       2009

  Data       14        28         41         55         69         83

  Monte      14        28         41         55         69         83
  Carlo                                                            
  ---------- --------- ---------- ---------- ---------- ---------- ----------

Table 16 Analysis CPU for the Far detector, Ntuple processing.

  ----------- ----------- ----------- ----------- ----------- -----------
              CPU in GHz                                      

  Year        2005        2006        2007        2008        2009

  Data        9           20          32          46          64

  Monte Carlo 9           20          32          46          64
  ----------- ----------- ----------- ----------- ----------- -----------

Table 17 Analysis CPU for the Near detector, Ntuple processing.

######## Candidate analysis

We assume 5 users each making 2 passes per year through the Candidate
data to create Ntuples. It takes 0.4 GHz-seconds/event to create an
Ntuple for the Far detector and 0.7 GHz-sec/spill for the near detector.
We assume for both detectors that this task is accomplished in 1 month
for the data and the Monte Carlo.

  ---------- --------- ---------- ---------- ---------- ---------- ----------
             CPU in                                                
             GHz                                                   

  Year       2004      2005       2006       2007       2008       2009

  Data       26        53         79         106        132        158

  Monte      26        53         79         106        132        158
  Carlo                                                            
  ---------- --------- ---------- ---------- ---------- ---------- ----------

Table 18 Analysis CPU for the Far detector, Ntuple creation from
Candidates.

  ----------- ----------- ----------- ----------- ----------- -----------
              CPU in GHz                                      

  Year        2005        2006        2007        2008        2009

  Data        28          62          100         150         200

  Monte Carlo 28          62          100         150         200
  ----------- ----------- ----------- ----------- ----------- -----------

Table 19 Analysis CPU for the Near detector, Ntuple creation from
Candidates.

In Table 20 we summarize the storage required for user Ntuples. We have
assumed that on average 1.5 passes will be stored.

  ---------- --------- ---------- ---------- ---------- ---------- ----------
             Data                                                  
             storage                                               
             (GB)                                                  

  Year       2004      2005       2006       2007       2008       2009

  Far                                                              

  Data       270       550        820        1090       1360       1630

  Monte      520       1040       1560       2080       2600       3120
  Carlo                                                            

  Near                                                             

  Data                 840        1850       3000       4400       6000

  Monte                840        1850       3000       4400       6000
  Carlo                                                            
  ---------- --------- ---------- ---------- ---------- ---------- ----------

Table 20 Storage required for user generated Ntuples.

Tape storage would be provided for all the user Ntuples. We expect to be
able to store all the Ntuples for users on disk.

## DATABASE

The database warehouse will be kept in Oracle and will reside on a SUN
Linux system. We will have a production machine and a development
machine. We plan to have Linux MySQL servers at Soudan and as replicas
for the offline analysis and the production farm. They will also be used
at collaborating institutions. We will require access to some fraction
of the Oracle client licenses, the number will be minimized by the use
of the MySQL replicas. The size of the database is expected to be about
500 GB over 5 years

## SUMMARY

  ------------------------ ------ ------ ------- ------- -------- ---------
  Year                     2004   2005   2006    2007    2008     2009

  Tape (TB)                6      39     60      35      41       162

  Disk (Ntuples) (TB)      0.8    4      5       3       3        13

  Disk (User Ntuples) (TB) 0.8    3      6       9       13       17

  Disk (DCache) (TB)       0.8    6      9       5       5        26

  CPU, Reco steady state   5      91     7       110     7        13
  (GHz)                                                           

  CPU, Reco reproc (GHz)   50     84     420                      1000

  CPU, MC (GHz)            120    315    63      560     63       130

  CPU, MC reproc (GHz)     12     600    900                      6500

  CPU, analysis (GHz)      80     154    170     590     200      230

  CPU Speed (GHz)          3.6    4.8    6       7.2     9.6      12

  Duals, Reco steady state 1      10     1       8       1        1

  Duals, Reco reproc       7      9      35                       42

  Duals, MC                16     33     6       39      4        6

  Duals MC reproc          2      62     75                       270

  Duals, analysis          12     17     15      41      11       10
  ------------------------ ------ ------ ------- ------- -------- ---------

Table 21 MINOS needs for tape, disk and CPU for 6 years of data taking.

Table 21 shows the resources required per year for 6 years of data
taking. This table shows the actual equipment that must be purchased
each year to meet the requirements described in the previous sections.
This takes into account equipment purchased in previous years. It also
assumes a 3 year replacement cycle for CPU. The CPU for reprocessing is
the amount required over a 3 month period for the years 2004-2006 and a
6 month period in 2009. Changing the assumptions about reprocessing will
change these amounts. Table 22 shows the costs associated with the needs
in Table 21. The following assumptions have been made. A dual CPU farm
node costs \$2200 independent of CPU speed. A 4 TB file server currently
costs \$14,400. We will assume that the cost remains roughly the same
but the capacity increases by a factor of 1.6 each year. We assume that
in 2004 the tape cartridge size is 200 GB and costs \$75. In 2005 and
beyond we assume that the capacity is 400 GB and costs the same. The
costs for the Monte Carlo CPU have been reduced by 50% on the assumption
that half the Monte Carlo is generated/reconstructed off-site. In FY2005
we expect to purchase a development database machine which will be an
additional \$30-40K. Upgrades to the disk on the database machines will
be needed, probably in FY07, but they have not been included here.

  --------------------- ------- -------- -------- -------- -------- --------
  Year                  2004    2005     2006     2007     2008     2009

  Tape                  2220    7400     11200    7000     8000     31000

  Disk (Ntuples)        2880    8300     7000     2300     1600     4000

  Disk (DCache)         2850    14300    13200    4000     3000     7800

  Disk (User Ntuples)   2850    7510     8500     8300     7600     5000

  Duals Reco steady     1500    21000    1200     17000    800      1200
  state                                                             

  Duals Reco reproc     15100   19200    77500                      92000

  Duals MC              18000   36000    6000     43000    36000    6000

  Duals MC reproc       2000    68000    82000                      300000

  Duals, analysis       25000   35000    31000    90000    28000    21000

  Total                 71500   217000   238000   170000   47100    463000
  --------------------- ------- -------- -------- -------- -------- --------

Table 22 MINOS costs in \$ per year for 6 years of data taking

## SUPPORT REQUIREMENTS

In addition to hardware needs, MINOS expects support from CD personnel
in a number of areas. There are two types of personnel, those directly
working on the experiment from the EXP department in Computing Division,
and personnel from other departments in the division that provide
support for MINOS.

Direct EXP support

1.  1 FTE to support the MINOS offline framework.

2.  1 FTE to support data handling operations including SAM, data
    archiving and related activities.

3.  0.5 FTE to support user desktops, machine purchases, installs of new
    equipment.

Support from other CD departments

1.  We are dependent on ROOT for our batch/interactive framework. We
    require support from the Fermilab ROOT team for MINOS-specific
    issues.

2.  We are using SRT (SoftRelTools) for configuration management. We
    require support for SRT from CD including responding to bug fixes
    and requests for additional features.

3.  We require system administration support from CSS for our Linux
    analysis farm and associated AFS disk.

4.  We will use ORACLE for our main database warehouse. We require an
    appropriate fraction of an ORACLE DBA be assigned to MINOS. We also
    require system administration of the Oracle database machines
    (production and development) and the LINUX MySQL servers. We also
    require a fraction of a database developer for issues related to our
    database interface software. We are using ODBC for our interface and
    require support for the odbc++ package.

5.  We are using ENSTORE and DCache for data archiving and data access
    and require timely resolution of issues that are relevant to MINOS.
    Off-site collaborators are using the dccp interface to access data
    read only without requiring a Kerberos ticket, we require this mode
    of access to continue.

6.  We plan to use SAM for our data handling and will require support
    from the SAMGrid team to deploy the system and help us debug
    problems.

7.  Our networks at the two detector sites have been installed by
    CCF/DCN. We require 24/7 support from them to monitor the network
    and fix problems when we are taking data. More details are provided
    in the section on NETWORKING SUPPORT.

8.  We are using the Control Room Logbook (CRL) as our electronic
    logbook. We require support for this including bug fixes and
    occasional requests for additional features.

9.  The DAQ systems are using Fermi LTS. As DAQ systems require
    stability and infrequent upgrades we may want to request extensions
    of support for particular versions depending on the MINOS data
    taking schedule.

10. We are using the central expwww webserver for our main experiment
    web server and we will continue to require support and problem
    resolution.

11. We are moving towards using GEANT4, initially for the beam
    simulation but also for the detector simulation. We will require
    support for use of GEANT4 such as new releases of the software and
    consulting help.

12. We are planning on using DocDB as a replacement for the existing
    NuMI notes database. The support agreement between the Program
    Support Group in the Computing Division at Fermilab and the
    NuMI-MINOS collaboration. As part of the production DocDB web server
    (cd-docdb.fnal.gov), Program Support will provide to NuMI/MINOS an
    instance of the current version of DocDB, which includes:

```{=html}
<!-- -->
```
1.  an alias on the DocDB server, minos-docdb.fnal.gov

2.  a backup of the database ( CSS will backup our dump files).

3.  a standard header and footer based on the current NuMI/MINOS
    website.\*

4.  Problem reporting for users: <DOCDB-ADM@fnal.gov> (mail general
    questions to Marcia Teckenbrock & John Urish), or submit a helpdesk
    ticket.

5.  script updates as required by product reconfiguration or upgrades.

> Requests for additional features or changes to DocDB should be
> submitted to <cd-proj-docdb@fnal.gov>. Such requests will be reviewed
> and depending on the availablility of resources and the feasibility of
> the requested change/s, will be added to this document.
>
> \* NuMI/MINOS will provide assistance if needed for implementation of
> certain technologies used for web headers/footers, if necessary.