**Testing of the HP G-Link Chip Set for an Event Builder Application**

Osamu Sasaki1, Jeffry Andresen, Hector Gonzalez, Masaharu Nomachi1, and
Ed Barsotti

Fermi National Accelerator Laboratory Document \# ESE-DAQ-940115

Batavia, IL. 60510

*Abstract*

The Hewlett Packard HDMP-1000 G-Link transmitter and receiver chip set
was tested for an event builder application. The re-lock time of the
serial link when a data path is changed is less than 30 µs provided fill
frames are transmitted. With a 1 kHz data path switching rate, this
results in less than a 3% data rate inefficiency due to
re-synchronization, which makes the HP G-Link chip set effective for use
in a large event builder for a hadron collider experiment.[^1]

**I. INTRODUCTION**

A large high energy physics detector requires a large scale data
acquisition system which could require several thousand data links
running at a speed of several tens of MBytes per second per link. A
transparent switch network with global traffic control being used as an
event builder for such a large experiment has already been proposed
\[1\]. The event builder receives event data fragments from many sources
via serial links. The transparent switch performs the event
reconstruction by the switching of the serial data links through the use
of global traffic control. One of the most important hardware parameters
of an event builder switch is the time that is taken to re-configure the
switch connections and to recover a synchronized clock to a new data
stream. This re-synchronization time needs to be a small percentage of
time in comparison to the switching interval to achieve sufficient data
throughput.

The Hewlett Packard HDMP-1000 G-Link chip set was selected to be tested
as it one of the candidates for a serial data link protocol chip set
\[2\]. The purpose of this test was to measure the re-lock time of the
HP G-Link chip set and to understand what other design parameters have
to be satisfied to build a switch based on using G-Links. The HDMP-1000
consists of a HDMP-1002 (transmitter) and a HDMP-1004 (receiver). The
transmitter serializes either 16 bits or 20 bits of parallel data, adds
4 coding bits, and transmits the data at a serial speed as high as 1.4
Gbps. The G-Link receiver converts the serial data to its original
parallel form. The G-Link chip set has three kinds of frames (data sets)
which are a data frame, a control frame, and a fill frame. The
transmitter sends fill frames if Data Available (DAV\*) and Control Word
Available (CAV\*) are false or if Enable Data (ED) is false. Data frames
and control frames are treated in the same way by the G-Link chip set.
Fill frames are sent by the transmitter at start up, whenever data
frames or control frames are not being sent by the user, and whenever
there is an error condition. A fill frame has the same duration as a
data frame or a control frame with one master transition to allow the
receiver to acquire frequency lock (both frame synchronization and bit
synchronization).

Once the receiver has frequency lock, data transmission can begin. The
4-bit coding field that is sent with the data has one master
rising/falling edge that the receiver\'s Phase Lock Loop (PLL) circuitry
uses to maintain bit synchronization as it recovers the clock from the
serial data stream. This phase lock has a narrow frequency detection
range. If phase lock is lost, fill frames are transmitted until re-lock
can occur through the use of frequency lock. It should be noted that the
receiver can never be re-locked when data or control frames are being
received. The G-Link system maintains DC balance on the serial data line
by inverting the transmitted data or control frame whenever necessary.
Figure 1 is HP\'s example of a simplex G-Link configuration.

![](media/image1.wmf){width="3.402083333333333in"
height="2.9159722222222224in"}

Figure 1: HP\'s Simplex Configuration Example

**II. TESTS**

*A. Testing Hardware*

This testing utilized a HDMP-1000 G-Link evaluation board simulating a
simplex configuration \[3\]. In the following tests, all serial signals
were transmitted over 50Ω SMA coaxial cable. The G-Link TX device on the
evaluation board was clocked from the STRBIN input by an external pulse
generator at 42 MHz. This frequency corresponds to a 1 Gbps bit stream
when the 20 bit data transfer mode is selected. The KEK PECL 4X4 switch
\[1\] was used to receive a 1 Gbps bit stream from the HP HDMP-1002 TX
device. PECL fanout/level adapter boards were used to adapt the G-Link
logic levels to PECL levels since the serial outputs from the TX are not
ECL but are buffer line logic (BLL) \[2\]. A clock divider board and a
NIM signal discriminator provided a 1 kHz NIM level pulse that was used
to change the KEK switch configuration. The bit stream exited the switch
and was sent to the HP HDMP-1004 RX device. A 20 bit random data
generator board provided a random data pattern when data was being
transmitted. The eye patterns of the serial bit stream and the receiver
strobe out signal were observed on a sequential sampling oscilloscope.
The re-lock time was measured by observing the Error and Link Ready
signals from the RX on a digital oscilloscope.

*B. Random Data Pattern Test*

A 20-bit random data pattern, along with the 4 coding bits that the
G-Link generates, was transmitted to the switch at 1 Gbps. As can be
seen in Fig. 2, the oscilloscope eye pattern clearly shows the 4 coding
bits along with the data bits.

![](media/image2.wmf){width="3.44375in" height="3.957638888888889in"}

Figure 2: Coding and Data Bit Eye Pattern

The coding bit field has a master transition of a single rising/falling
edge which serves as a fixed timing reference for the G-Link receiver\'s
clock recover circuit. For this test, in order to send a continuous
random data pattern regardless of the lock condition, DAV\* and ED were
always enabled through the use of jumpers. In a non-switching mode, the
G-Link chip set transmitted and received the 20-bit data pattern without
losing lock. When the switch was in its 1 kHz switching mode, the G-Link
receiver would lose lock and never re-lock. For the RX G-Link to
synchronize with the TX G-Link when only one clock source is used in the
simplex configuration as shown in Fig. 1, the G-Link system requires the
TX to send fill frames to the RX G-Link.

*C. Fill Frame Phase Shift Test*

In this test, the G-Link evaluation board continuously transmitted and
received fill frames. The phase of the serial signal was changed along
with the timing of the changing of the switch in relation to the fill
frame. The top scope trace in Fig. 3 shows the G-Link fill frame.

![](media/image3.wmf){width="3.3743055555555554in"
height="4.165972222222222in"}

Figure 3: G-Link Fill Frame

There are two types of fill frames \[4\]. At startup, fill frame FF0 is
transmitted which has a single falling edge in the data field going from
a high D9 to a low D10. Once frequency lock occurs, fill frames FF1L and
FF1H are transmitted. With the FF1 fill frames, the position of the
falling edge in the data field is shifted forward or backward by one
bit. This is accomplished by toggling data bits D9 and D10. FF1L
transmits zeros for D9 and D10, and FF1H transmits ones for D9 and D10.
The transmitter sends either FF1L or FF1H to reduce the cumulative
serial DC offset. FF0 maintains DC balance as it is a square wave with a
50% duty cycle. The rising edge of the fill frame is used by the
receiver to achieve frequency lock. The lower scope trace in Fig. 3 is
the receiver\'s strobe output (RSTRBOUT). This is the clock that has
been recovered from the receiver\'s serial input. The TX serial link was
connected to the input of a PECL fanout board. Figure 4 is a block
diagram of the test setup.

![](media/image4.wmf){width="3.4159722222222224in"
height="1.7770833333333333in"}

Figure 4: Cable Delay Test Configuration

The PECL KEK 4X4 switch was used as a \"2 to 1 SELECTOR\" which changed
the signal path every millisecond. The serial data cable lengths from
the fanout to the switch were varied to create signal delays. The delay
was incremented in steps of 5 ns for the serial signal from the fanout.
Delays in 4 ns increments were added to the 1 kHz signal for the
selector switching to change the point in the fill frame at which the
changing of the switch occurs. There are two parameters affecting the
re-lock time in the configuration of Fig. 4. The first is the switch
changing the signal path which shifts the phase of the serial signal.
The second parameter is where within the fill frame the changing of the
switch occurs. The re-lock times were measured as a function of these
two parameters. Figure 5 is an example of a re-lock time reading.

![](media/image5.wmf){width="3.5131944444444443in"
height="2.9715277777777778in"}

Figure 5: G-Link Re-Lock Time

To begin the test, all serial data cables were the same 1 meter length.
The G-Link evaluation board operated without losing lock which indicated
that the switching did not create errors when the phase difference
between the two serial signals is zero. The switching from a 0 ns delay
to a 24 ns serial data delay results in a zero phase shift as the frame
has a 24 ns period. The system did not lose lock when this 24 ns delay
was tested. The system did lose lock as the 5 ns delay steps where
introduced. The re-lock time was as short as 8.1 µs and as long as 27.4
µs. Adding delay to the 1 kHz NIM signal did not affect the maximum
re-lock time. Figure 6 is a graph of the re-lock time in relationship to
the delay time.

![](media/image6.wmf){width="3.44375in" height="2.3048611111111112in"}

Figure 6: Re-Lock Time vs. Phase Change

Figure 6 shows that if the clock is in the correct phase, the G-Link
operates without losing lock. The negative time readings were when the
switch went to a shorter cable length. The maximum re-lock time did not
occur when the phase difference was the greatest. The -5 ns and 20 ns
points are sensitive points in that either a small re-lock time or the
maximum time occurs at these two points. The -10 ns and 15 ns delays
also produced two different re-lock times, but the difference between
the times was smaller. The re-lock times shown in Fig. 6 follow a
pattern that repeated in succeeding frames.

*D. Fill Frame Frequency Change Test*

Figure 7 is a diagram of the frequency change test.

![](media/image7.wmf){width="2.6798611111111112in"
height="1.7215277777777778in"}

Figure 7: Frequency Change Test

This test did not use the G-Link TX device. The first clock generator
was connected to one input of the switch with a fixed frequency of 42
MHz. A second clock generator was connected to another input of the
switch with a frequency that was changed in incremental steps on both
sides of the 42 MHz frequency of **fo**. Since the G-Link RX device
recovers its clock from the serial stream, the 32 MHz to 58 MHz
frequencies were chosen to be in the same frequency range as the fill
frame rate that the RX can receive. In other words, the two clock
generators transmit similar serial signals as that of the fill frame
signals of the TX. The G-Link RX device did not lose lock when the two
switch inputs were at the same frequency of 42 MHz without a phase
difference. Figure 8 shows that as the difference in frequency
increased, the re-lock time increased.

![](media/image8.wmf){width="3.44375in" height="2.6798611111111112in"}

Figure 8: Frequency Change Re-Lock Time

The sequence of switching from **fo** to **f** or from **f** to **fo**
resulted in the same re-lock times. By tying the **f** input of the
selector low, the time for the RX\'s PLL circuit to re-lock from its
disabled state was measured. The result was 1.8 ms.

*E. Simplex Dual Clock Data Pattern Test*

In the simplex configuration with one clock generator as shown in Fig.
1, when RX unexpectedly loses lock, RX can not re-lock without receiving
fill frame signals from the TX. This configuration does not provide the
TX a way of knowing that RX lock has been lost and that fill frames need
to be transmitted. Figure 9 is a diagram of a simplex data pattern test
with dual clocks that was done. The RX in this dual clock simplex
configuration has the ability to recover the lock by itself.

![](media/image9.wmf){width="3.4159722222222224in"
height="1.7215277777777778in"}

Figure 9: Simplex Dual Clock Data Pattern Test

In this test, a 1 Gbps data pattern (a data frame) was sent from the TX
G-Link to the RX G-Link. The TX and the RX have dual ports for the
serial signal. The Loopen signal controls whether the Dout or Lout
output and the Din or Lin input are currently enabled. When Stat1 is
low, Dout and Din are active. Clk2 provides the frequency pattern which
simulates a fill frame pattern for the receiver to lock upon. When lock
occurs, Stat1 goes high which activates Lin instead of Din. The TX Lock
output was connected directly to the TX Loopen input. A fixed precision
crystal clock was used for the transmitter clock. A HP 8110A pulse
generator was used as the variable receiver clock as it has better than
0.1% stability, period steps of 10 ps, and duty cycle steps of 0.1%. The
fixed clock was disabled and then enabled with a second pulse generator.
When the TX clock is disabled, TX Lock and TX Loopen go low which
disables the Lout and Lout\* outputs. This results in the RX losing lock
and switching to the Din input. The RX clock then supplies a fill frame
like clock for the receiver to re-lock upon. Fixed clock frequencies of
40.0000 MHz and 25.002 MHz were tested with Fig. 10 showing the 40.0000
MHz re-lock times. The 25.002 MHz test produced similar results.

![](media/image10.wmf){width="3.152083333333333in"
height="2.7631944444444443in"}

Figure 10: Simplex Test Re-Lock Times

Figure 10 indicates that the two clocks need to be within 0.1% of having
the same clock frequencies. This result confirms HP\'s specification
\[6\] and agrees with the test that was done at LBL \[5\]. The points at
the top of the graph are actually points going beyond the graph
indicating that re-lock did not occur at those test points. When the
same clock generator was used for the transmitter and receiver in the
Fig. 9 setup, there was no difference in frequencies with re-lock never
occurring. When the two different clocks were within a 0.1% difference
in clock frequency, the G-Link regained lock provided the duty cycle of
the variable RX clock was 50%. However, there were duty cycles that
would cause the RX G-Link to never re-lock. Figure 11 is a diagram of
the test setup that was used to test the duty cycle of the receiver\'s
clock.

![](media/image11.wmf){width="3.332638888888889in"
height="1.4159722222222222in"}

Figure 11: Simplex Receiver\'s Clock Duty Cycle Test

The HP 8110A pulse generator provided a stable clock at the selected
test frequencies. The duty cycle of the clock was changed in 0.1% steps
from a 20% duty cycle to a 80% duty cycle. Figure 12 shows the
frequencies that were tested and the duty cycle ranges in which the RX
G-Link would not re-lock.

![](media/image12.wmf){width="2.861111111111111in"
height="2.611111111111111in"}

Figure 12: Duty Cycle Error Ranges of G-Link RX Clock

The test results in Fig. 12 show that there is a narrow duty cycle range
on each side of 50% for each frequency in which the G-Link receiver will
not re-lock. When this same test was repeated except that the duty cycle
of the G-Link transmitter clock was varied, the G-Link never lost lock.
The G-Link transmitter compensates for its clock not having a 50% duty
cycle. The inability to re-lock occurred in 16-bit mode and 20-bit mode
and is when the G-Link is used in the two clock simplex mode. This is
important as the duty cycle of a clock used in an actual G-Link data
system may not be exactly 50%. Further testing needs to be done on newer
versions of the G-Link chip, set including the HDMP-1012 transmitter and
HDMP-1014 receiver, to verify if the re-lock time is still sensitive to
the clock duty cycle.

**III. Conclusion**

In an event builder switch setup, it is required that a serial data link
system have the ability to regain lock within a reasonable time on the
change of the switch configuration. Our tests demonstrate that the
G-Link can re-lock in less than 30 µs provided fill frames are sent. In
the application of the event builder, the single clock simplex mode can
be used if the TXs are forced to send fill frame signals on each change
of the switch configuration. With an expected minimum switching interval
of 1 ms, devoting a period of 30 µs to send fill frames to assure
re-lock would result in only a 3% data throughput inefficiency. In the
dual-clock simplex mode, re-lock performance is critically dependent on
relative clock frequencies and the receiver\'s clock duty cycle, which
makes this mode unsuitable for the event-builder application.

**IV. Acknowledgments**

We would like to thank J. Butler, Prof. S. Iwata, T.K. Ohska, and Y.
Watase for their support.

**V. References**

\[1\] O. Sasaki, M. Nomachi, T.K. Ohska and H. Fujii, \"A VME Barrel
Shifter System for Event Reconstruction for up to 3 Gbps Signal
Trains\", IEEE trans. Nucl. Sci., NS-40 (1993) 603.

\[2\] \"Gigabit Rate Transmit Receive Chip Set Technical Data\", Hewlett
Packard.

\[3\] \"Reference Guide for the Gigabit Rate Transmit/Receive Chip Set
(HDMP-1000) Evaluation Board\", Hewlett Packard, October 1992.

\[4\] \"G-Link: A Chipset for Gigabit-Rate Data Communication\",
Hewlett-Packard Journal, October 1992.

\[5\] B. Turko, M. Wong, \"Measurements With External Clock At Receiver
To Re-Establish Lock in Simplex Mode\", LBL, March 1993.

\[6\] \"Low Cost Gigabit Rate Transmit/Receive Chip Set\", Hewlett
Packard, November 1993.

[^1]: 1 National Laboratory for High Energy Physics (KEK)

    Tsukuba, Ibaraki-Ken, 305 Japan