# Presentation: 212200

## L2Alpha Status

- R. D. Martin
- UIC

## All bare boards manufactured
Only 3 stuffed as preproduction
preproduction boards delivered 21 Feb 2000
Good news:
newly added circuitry works
outside of cache, only a couple jumper wires and a couple other corrections.
Bad news:
The Level3 (off-chip) cache

- Only 3 stuffed as preproduction
- preproduction boards delivered 21 Feb 2000
- Good news:
- newly added circuitry works
- outside of cache, only a couple jumper wires and a couple other corrections.
- Bad news:
- The Level3 (off-chip) cache

## Cache organization

- 3 levels of cache
- 2 on-chip
- split L1 cache
- 96kB 3-way set associative L2 cache
- consider as 3 32kB cache
- L3 cache is off-chip
- 4MB, uses 7.5ns SRAM
- L2, L3 cache line is 64 bytes

## L3 Cache Problems

- 1 “works” = compile linux kernel w/o crash
- 30 min heavy paging; **crash if bad pointer**
- linux: crash on detected (even if corrected) or ignore
- “working” is still marginal; had to adjust settings
- no parity errors detected with adjusted settings
- 2 don’t work by this criterion
- regularly have (corrected) parity errors
- error rate >105 worse than PC164 (Compaq)
- >1 Hz during memory stress test
- 2-bit errors probably too high for random
- Far too many to crash on detection of even doubles
- can’t provoke at will by program we’ve written
- one “worked” with 4 SIMMs (1/2 memory BW)

## Characterization:(Believe it’s still a layout problem)

- ECC errors from 1 of 4 data switches
- 8 of 10 worst lines same in all 3 boards
- Impedance?  	(trace geometry, reflections)
- Cross-talk? (replace some traces with wires?)
- two bad lines **better with a wire** with pins pulled than on its intended trace
- two bad lines **better with a wire**  running only between cache and data switch vias
- Timing problem?	(try lower speeds)
- may help in diagnosis
- not a silver bullet: board just somewhat better...
- study temperature, voltage sensitivity
- how dangerously marginal?  OK w/o L3 cache?
- consulting engineer to help study signal problems?
*Compaq 21264 (harder) : problems tho better layout*

## Characterization (continued)

- study temperature, voltage sensitivity
- how dangerously marginal?  OK w/o L3 cache?
- consulting engineer to help study signal problems?
*Compaq 21264 (harder) : problems tho better layout*

## Workarounds?

- Run w/o L3 cache  (smaller L3 cache no help)
- data (~3KB max) must come in every event
- L2, L3 caches work with 64B granularity
- 128MB to 64MB? (memory bandwidth/2)
- enough yield to be useful? Didn’t “fix” all 3 boards...
- Board selection?
- low performance boards to light load (Admin?)
- enough yield of cache-ready boards or slow-mem???
- Slower clock?   Only better, not solid...

## Remediation

- New layout?
- Use more of information from Digital’s layout?
- Used chip topology, but not layout details
- Different cad system; diagnostic port different
- Fix problems understood(?) from this iteration?
- Still physicist doing layout at UM
- mostly auto-router
- several months per cycle
- external layout/consultant?
- Lead time? Co$$$t?

## Partial production?

- Make enough boards to commission
- designate as administrators/test stand?
- Need to be sure Admin’s stay lightly loaded...
- Iterate design before use rest of parts
- several parts are obsolete
- believe own parts for 10-20 than current plan
- more $$$

## Cache problem

- Only see failures when booting linux or compiling the linux kernel
- Cannot reproduce failures with code intending to thrash the cache
- See ~thousand single bit ECC errors on roughly 10 bits when running simple memtest
- Assume failures due to uncorrectable double bit errors