# Presentation: 676814

## Networking and Computer Support at SUL

- Liz Buckley-Geer, FNAL

- Review of Soudan Operations

- May 14, 2003

## Outline 

- Network configuration and support
- Computing systems and support
- Conclusions

- Review of Soudan Operations

- May 14, 2003

## Networking at SUL

- Review of Soudan Operations

- May 14, 2003

## Network Configuration and Support

- There are two LANS at SUL
- The existing  SUL LAN which supports the Soudan II computing, some MINOS desktops and some CDMS systems. The SUL LAN is supported locally at SUL with no FNAL CD involvement
- The FNAL/CD-supported LAN which supports the majority of the MINOS and CDMS computing and is divided into a MINOS LAN and a CDMS LAN

- Review of Soudan Operations

- May 14, 2003

## Network Configuration

- The FNAL supported network at SUL is logical extension of the Fermilab campus network via GRE (generic routing encapsulation) tunnel;
- Physical connection is via T1 (1.5Mbps) line of the University of Minnesota  to the UMN campus at Duluth
- 100Mbps backbone formed  by the Cisco7505 router and  2 Cisco Catalyst 4006 switches South and North, several  workgroup switches C2924, C2950 are in use also.
- Switches are partitioned to make  5 subnets DAQ,  DCS, DHCP, Servers  and CDMS with routed block of IP addresses  and one private network for CDMS. All public subnets are terminated in the **r-s-soudan** router. Access policy between subnets and the external world is implemented at the border by reflexive ACLs (access control lists). Reflexive lists allow incoming traffic for sessions initiated from onsite.
- CDMS subnets are extended to the surface

- Review of Soudan Operations

- May 14, 2003

## Network Configuration

- DNS/DHCP, KDC and network monitoring  are supported via local servers tied to the central facilities at Fermilab and capable of providing service even if connectivity to FNAL is broken
- Current configuration of the switches: ~350 10/100 ports, ~150 node connected. Additional line cards  could be installed to provide more 10/100/1000 Ethernet connections

- Review of Soudan Operations

- May 14, 2003

## MINOS LAN Configuration

- The network domain is minos-soudan.org (198.124). There are four pools of addresses at SUL

| DHCP POOL | Visible to general Internet | 191.124.212.0 with addresses from 1-199 and 202-253
 |
| --- | --- | --- |
| Server LAN | Visible to general Internet | 198-124-213.0 with addresses from 1-29
 |
| DCS LAN | Behind firewall | 198.124.213.64 with addresses from 65-125
 |
| DAQ LAN | Behind firewall | 198.124.213.128 with addresses from 129-253
 |

- Review of Soudan Operations

- May 14, 2003

## The DAQ and DCS LANs only reachable from the outside world via a fully kerberized “gateway” node called minos-gateway which is connected to the Server LAN. This machine has 3 ethernet interfaces.
All machines on the Server and DHCP LAN are required to run kerberized services such as ftp,ssh. The hourly compliance scans run by FNAL Computer Security includes the Server and DHCP LANS.
We require users at SUL to have FNAL computer accounts and kerberos principals before we will give them an account on a MINOS computer.

- All machines on the Server and DHCP LAN are required to run kerberized services such as ftp,ssh. The hourly compliance scans run by FNAL Computer Security includes the Server and DHCP LANS.
- We require users at SUL to have FNAL computer accounts and kerberos principals before we will give them an account on a MINOS computer.

*MINOS LAN Configuration*

- Review of Soudan Operations

- May 14, 2003

## CDMS LAN Configuration

- FNAL CD operates two subnets for CDMS, cdms-soudan.org and cdms-private.net
- both networks extend to the CDMS surface trailer via fiber
- cdms-private.net is critical for all DAQ operations
- CDMS also has machines connected to the U. Minn. network (umn.edu), supported by SUL staff
- umn.edu are used for data analysis, taking advantage of the University’s MATLAB license

- Review of Soudan Operations

- May 14, 2003

## CDMS Network Issues

- Network Issues:
- CDMS data rates are too high (~ 1 MB/sec background, ~ 10 MB/sec calibration) to allow data streaming to FNAL over the public network
- They intend to maintain a significant analysis farm on the surface, and therefore need good network performance (fast ethernet now, gigE future) over the link to the surface

- Review of Soudan Operations

- May 14, 2003

## Future network plans

- Both MINOS and CDMS would like to be able to operate their detectors from the surface building
- This involves getting fiber from the headframe to the surface building
- Investigations are going on with the DNR and Minnesota Power on the possible cost.
- It would also be very nice to upgrade the speed of the connection between FNAL and SUL. Currently we have about 150 Kbytes/sec transfer rate between the two sites. The cost of doing this is not known.

- Review of Soudan Operations

- May 14, 2003

## MINOS Computing Systems

*MINOS Computing Systems*

**minos-crl**

**minos-offline**

**minossrv**
**W2K server**

**dcsdcp**

**dcs**

**minos-gateway**

**Server LAN**

**DAQ LAN**

**DCS LAN**

- NFS

- NFS

- Data to
- FNAL

- Review of Soudan Operations

- May 14, 2003

## MINOS Computing Systems

- Server LAN – all Linux machines except minossrv
- minos-gateway – used for offsite access to the restricted DAQ and DCS LANs, has 3 ethernet interfaces
- minos-offline – serves database, used for general offline analysis by physicists at SUL
- minos-crl – runs Control Room Logbook and web server
- minossrv – Windows 2k server
- DAQ LAN – all Linux machines
- daqrc – Run Control
- daqdcp – writes raw data to disk, archives it back to tape robot at FNAL, creates DAQ status page and puts in on NFS mounted minos-crl disk
- daqdds – online monitoring and event display
- daqdev – online development machine
- daqtpc – timing control
- BRPs and Trigger farm – front end systems connected via PVIC interconnect

- Review of Soudan Operations

- May 14, 2003

## MINOS Computing Systems

- DCS LAN – all Linux except node dcs
- dcs – W2K machine for environmental and coil monitoring
- dcsdcp – writes DCS raw data and archives it to tape robot at FNAL
- HV monitoring – 2 machines
- Magnet monitoring – 8 machines
- DHCP LAN
- Various user laptops
- Desktops
- 4 Windows machines – 2 on the MINOS LAN and 2 on the SUL LAN
- One laptop
- 4 machines in SUL counting house on SUL LAN

- Review of Soudan Operations

- May 14, 2003

## CDMS Computing Systems

- Current systems
- 6 machines dedicated to DAQ
- 3 machines to the dilution refrigerator
- 6 machines for analysis (surface trailer)
- DAQ and refrigerator systems are self-maintained, with occasional assistance from SUL staff (Dave Saranen, Jerry Meier)
- Analysis systems are self-maintained

- Review of Soudan Operations

- May 14, 2003

## Computing Support

- Current operations
- One on-site sysadmin (Dave Saranen) who is responsible for supporting computers and liaising with FNAL networking group
- Remote support by members of the FNAL CD/EXP dept (MINOS), FNAL CD/CCF dept (CDMS) and the RAL MINOS DAQ group
- Planned operations – starting summer 2003
- Jerry Meier will be taking over as the on-site sysadmin. He will need to come up to speed on Linux support – Soudan II systems were all VMS. We plan to have him come to FNAL and take our Linux sysadmin course
- He will support the MINOS and CDMS systems
- Remote support by members of the FNAL CD/EXP dept (MINOS), FNAL CD/CCF dept (CDMS) and the RAL MINOS DAQ group will continue

- Review of Soudan Operations

- May 14, 2003

## Conclusions

- Support of network by FNAL/CD is working well
- Both experiments would like to have control rooms in the surface building. Need to determine the cost of bringing fiber from the headframe to the building
- Remote support of computing by FNAL CD and RAL + local sysadmin should be sufficient, once Jerry is freed from most of his current responsibilities and has come up to speed

- Review of Soudan Operations

- May 14, 2003