# Presentation: 295195

## EVM Project Status

**	****S. Aziz, M. Litmaath, C. Moore, V. O’Dell, I. Suzuki**
**	****	****	****	****	****FNAL**
**	****	****S. Pavlon, K. Sumorok**
**	****	****	****	****	****MIT**
**	****	****Topics**
**RCN**
**EVM  Test Bench**
**EVBSIM results**
**XDAQ Run Control**
**Future work** **	****	**
**	**

## Readout Control Network

**Implemented on Linux with**
- 1) Ethernet UDP/IP
- 2) IEEE1394 asynchronous broadcast transmission
- 	normally point to point with confirmation
- 3) IEEE1394 isochronous broadcast being worked on
- 	IEEE1394 limited to 63 nodes with current technology
- 	Bridges should be available in the Future
- 	Data transmitted in a 125 us cycle > 8K blocks/s
- 	Currently 400Mb/s data rate
- 	Broadcast mode delivery not guaranteed

## Reliable Broadcast Protocol

**ID information from EVM to RU  broadcast in Packets**

-  NACK based
-  Receiver side packet loss detection
-  PacketX XOR of original    packets
-  ACKs from few clients as  a pacemaker

## Reliable Broadcast Library

-  Implemented in C++
-  Modular design
-   Capable of Using both
-   Ethernet and IEEE1394
- Preliminary Benchmark
-   Tests done
- Protocol Specification
-   Document revised

- http:/home.fnal.gov/~ichiro/document/evm-rcn/main.ps

## Benchmark Test of Library

**Test Conditions**
-  Two quadlets of EVT-ID data
-    per Trigger
-  UDP/ID  on FastEthernet or
-   FCP(async) on IEEE1394
-  No Forward Error Correction
-    No Transmission error
**Maximum Rate**
-  UDP: 1300kHz (10.5 MB/s)
-  1394:  600 kHz (5.0 MB/s)

## Things to be done

**RCN integration into ****XDAQ**

-  Add peer-transport for the RCNP
-  Add Callback methods in the RUI,RM
-  Modify EVT-ID Management in RUI-RUO, RM-BM

## EVM Test Bench

**EVM, RU and BU Driven by XDAQ**

- Myrinet used
- for both BCN
- and BDN in
- tests

## Modules – Brief Description (1/3) 

**L1**
- Sends triggers to EVM and DDU (RU)
- Receives throttle commands from EVM
**EVM**
**Receives**
- Triggers from L1
- Event requests and releases from BU via BCN
**Sends**
- Throttle commands to L1
- Triggers to RU via RCN
- Event IDs to BU via BCN

## Modules – Brief Description (2/3)

**RU (emulates 1 DDU)**
**Sends** data fragments to BU via BDN
**Receives**
- Triggers from L1 (DDU)
- Triggers from EVM via RCN
- Data fragment requests from BU via BCN
**BU (emulates N Filter Nodes each with M CPUs)**
**Sends**
- Event requests and releases to EVM via BCN
- Data fragment requests to RU via BCN
**Receives**
- Event IDs from EVM via BCN
- Data fragments from RU via BDN

## Modules – Brief Description (3/3)

**BCN (no internal structure – NICs do the work)**
- Sends control messages to EVM, BU, RU
- Receives control messages from BU, EVM
**BDN (no internal structure – NICs do the work)**
- Sends data fragments to BU
- Receives data fragments from RU
-  **XDAQ does not distinguish BCN and BDN**
-  EVBSIM uses BCN also for data fragments

## EVM Simulation

**EVBSIM new components**
- BCN and BDN have no internal structure
- BCN and BDN use NIC objects for traffic sequencing
- NICs ensure each endpoint does only one
-   send/receive/DMA at a time
- NIC can send and receive at the same time
-   (easy to change if ever necessary)
**Problems**
- 500x500 cannot keep up with trigger rate:
- RU does not implement barrel shifter yet (not trivial)
- BU traffic shaping helps, but seems insufficient
- 500x500 memory usage >~ 2 GB !
- There might still be a leak somewhere

## EVBSIM Results (1)

**Comparison of 1x1 test bench varying**
-  Number of event requests per packet (64 bytes per event)
-  Number of concurrent event request packets in BU
**Results tuned by varying**
-  DMA from host memory to NIC	10 us + bytes/132 us
-  Myrinet transfer			  5 us + bytes/160 us
-  DMA from NIC to host memory	15 us + bytes/132 us
-  **224 byte I2O header**
-  Smallest message  224 + 64 = 288 bytes

## EVBSIM Results (2)

**Bandwidth EVMSIM/XDAQ in MB/s**
-              Events per Packet
-    	1	 	4		16		64
-   10/10 = 1.0	  25/26 = 1.0	  43/44 = 1.0	  52/53 = 1.0
-   23/23 = 1.0	  43/49 = 0.9	  55/58 = 0.9 	  59/61 = 1.0
-   24/25 = 1.0 	  43/49 = 0.9	  55/58 = 0.9
-   24/25 = 1.0

- Concurrent
-  packets

- 1
- 4
- 16
- 64

## EVBSIM Plans

**Simple model OK for 1x1**
- Let’s see how it does for 2x2, 1x3, 3x1, etc.
- March, April
- Extract trends, fudge factors, if any
**Implement RU barrel shifter for LHC case**
- See if it allows the DAQ to keep up with L1
- Use simple BCN for now

## XDAQ Run Control Status

**XDAQ Run Control Status**

**Learning Curve for jxdaq java code**
- Building code
- Booting components
- Shutdown of xdaq when using ssh instead of rsh
**Ability to Automate xdaq and obtain results**
- Currently using a perl script with xdaq Tcl interface
- Desire same functionality but via xml/xdaq interface.
**First pass at automating xdaq**
- Modified jxMain such that GUI appears but no mouse clicks required for actions(e.g. Open, boot and configuring IOP’s). Results still displayed on GUI. Works but is overkill for an automation.

## Future work

**IEEE1394 bandwidth**
-  Isochronous transmission
-  Modification of the Linux driver
**Test Stand**
-  Benchmark test of 2x2 and EVBSIM comparison
-  Integration of RCN into XDAQ
-  Implementation of XDAQ Run Control GUI.
-  Automated scripts for XDAQ with Command line Interface
**EVBSIM**
-  Implement RCN XDAQ modifications
-  Extend EVBSIM to NxN
**Technical Design Report (TDR)**