<!-- Title: 16:34:48  Mon Oct 15 2007 -->
<!-- Crewchief: Usual Suspects -->
<!-- Op1: &nbsp; -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>The shutdown is ending.  Start a new page.</pre>


<!-- Date: Mon Oct 15 16:56:02 2007 -->
Today Eric and I swapped many TDCs in the hall.  We were out of Rev F spares, so we put Rev C boards from spares into slots 13-17 of COT08 in order to generate Rev F spares. 
<p> 
Then we swapped out boards with problems: 
<ul> 
<li><b>cot07, slots 14 and 16-</b> These had large timing shifts (11ns).  The shift was prone to shift by 6ns on a power cycle. 
<li><b>b0cot00, slot 3-</b> Needed to be hot-seated on a power cycle 
<li><b>b0cot00, slot 15 and b0cot10, slot 17 -</b> These two (rev F) boards had 48 out-of-time t0s in Eiko's calibration 
</ul> 
 
Finally, we moved 3 boards that don't work for XTC to the EM 
timing crate.  Now only two slots of that crate are have cards 
that are not known to be bad for XTC. 
 
The swaps are as follows.  XTC grade is listed for boards taken from spares: 
<pre> 
Crate   Slot  Old  New  XTC  Reason 
COT00    3    108  681  C    power up failure 
        15    901  857       48 chan. t0 shift 
COT07   14    504  503  B    11 ns t0 offset 
        16    690  703  D    11 ns t0 offset 
COT08   13    923  519  D    Need Rev F spare 
        14    823  562  C     "    " 
        15    868  537  C     "    " 
        16    853  327  A     "    " 
        17    854  164  A     "    " 
COT10   17    882  853       48 chan. t0 shift 
ETDC00   6    715  329  F    No XTC 
         7    609  324  F    "   " 
        15    517  730  F    "   " 
</pre> 
Once the XFT test stand is back, we'll need to grade the boards from the EM Timing crate and configure at least one as a stereo spare.  Also, one of the rev F boards taking from the hall should be made into an axial spare.   
<p> 
The boards that need to be reseated after power up should be retested.  Assuming they are still OK on the test stand, we should move them to the HTDC crate.  They should be OK for trigger since they worked with XTCs.  It is much less of a pain to reseat those if that crate needs to be power-cycled.
<!-- Author: jdl -->


<!-- Date: Mon Oct 15 17:07:12 2007 -->
<p> 
TDC 653 was removed from b0cot09, slot 2 on 9/7/07.  The problem came on power-up and is described <a href= 
'/cgi/elog/elog.pl?nb=2007&action=view&page=719&frame=2&anchor=16:25:46.706'> 
in the shift e-log</a>.  It should be retested. 
<p> 
TDC 618 was jammed into the XFT test crate last week.  Remnants of the destroyed backplane connecter fell out of the card's J0 connector.  Both the J0 and J3 are suspect.  They should be examined and probably be replaced. 
<p>
<!-- Author: jdl -->
<!-- Comment: Wed Oct 24 10;37;46 comment by...jdl -->
Oops.  It was 613, not 618.
<!-- Comment: Wed Oct 24 10;45;58 comment by...Lou DalMonte -->
<pre>TDC 613 has had the P0 and P3 connectors replaced. Card passes testing at the TDC Teststand.  LD

TDC 653, no problems found testing at TDC Teststand.  LD</pre>
<!-- Date: Tue Oct 16 12:01:21 2007 -->
TDC 324 was not reading out in the EM Timing crate.  Bunch counter was stuck at 118.  I tried reseating the card and rebooting the crate.  No change.  I put back TDC 609 that had been in that slot before.  It works OK.
<!-- Author: jdl -->
<!-- Comment: Thu Feb 28 13;35;28 comment by...Lou DalMonte -->
<pre>TDC 324 Failing RC. All Channels fail. (no done Timeouts). Same results testing on 2/21/08 , 2/27/08. All Mutrig functions appear normal. No clue at this time as to what the problem is.  LD</pre>
<!-- Date: Wed Nov 7 16:02:28 2007 -->
<pre>See <a href= 
'/cgi/elog/elog.pl?nb=2007&action=view&page=866&frame=2&anchor=16:00:02.635'> 
this entry</a> about TDC and XTC swaps today.  The time offset board 551 was replaced with 613 in COT17, slot 13.  Board 551 has been put aside for further investigation. 
</pre>
<!-- Author: jdl -->


<!-- Date: Mon Jan 14 13:29:52 2008 -->
<pre>I swapped out the TDC in b0cot08 slot 6 that was intermittently failing config for the XTC2.   
<ul> 
<li>Old: TDC-689, XTC-151 
<li>New: TDC-632  XTC-53 
</ul><p></pre>
<!-- Author: jdl -->


<!-- Date: Tue Jan 15 17:02:48 2008 -->
<h3>TDC 339</h3> 
 
Lou and I have been looking at this TDC, which was giving the following errors in the RC test:<br> 
<pre>*** TDC_setup: VME bus error reading slot 16 channel 5 calibration reg [2] ***</pre> 
Turns out that these 
are not real vme bus errors, bit 0 of that register (0x2041800) 
is toggling between 0 and 1. Today we checked it with the 
LA and saw that during VME reads the bit changes (the trace 
below shows such a cycle, where a 0 should have been read, but we got a 1 instead). Since Lou swapped the FPGA already, we think that that's not the problem, but we'll have to check.<br> 
<br> 
<a href='/elog/notebooks/tdc/Tue_Jan_15_2008_145523.ps'><img src='/cgi/elog/det-system-elog.pl?nb=tdc&action=view&page=-39' ></a><br>
<!-- Author: Lou/Sebastian -->
<!-- Comment: Wed Jan 16 14;52;41 comment by...Lou/Sebastian -->
<b>Lou found a no-connect in the FPGA. So he connected pin2 to pin11 (netname DATA0). This is consistent with the other TDCs and with the schematics. We tested the fix and it now works! However it looks like there is an unrelated problem with channel 18. </b>


<!-- Date: Tue Feb 12 15:37:41 2008 -->
The vme crate of the tstmuon1 teststand was swapped today. No problems with the crate so far.
<!-- Author: Sebastian -->


<!-- Date: Wed Feb 13 12:13:05 2008 -->
We made two TDC swaps.  Both were for XFT problems. 
<bl> 
<li>b0cot00, slot 14 (SL2): S/N 913→894.  XTC was dead 
<li>b0cot11, slot 13 (SL5): S/N 721→525.  XTC2 had reduced efficiency 
</bl> 
The TDCs should be rechecked with new XTCs and returned to spares.
<!-- Author: jdl -->
<!-- Comment: Thu Feb 14 14;23;00 comment by...Sebastian -->
TDC 899 was taken out of cot04 slot8 (was giving constant DoneTOs). Lou will look at it.

<!-- Comment: Thu Feb 21 11;04;17 comment by...Lou DalMonte -->
<pre>TDC 899 test results: Done Timeout Err on very first RC pass  only. Never repeated this failure again, after many tries over a period of a few days. Remove XTC2 board SN 47. The idea is to use this board on the first floor for now. -LD
</pre>

<!-- Comment: Thu Feb 21 11;10;23 comment by...Lou DalMonte -->
<pre>TDC 913 passed all tests. Grade for XTC -> A.
    then matched to new XTC1 SN 47.
    Return to spares.
TDC 721 passed all tests. Grade for XTC -> C.
    then matched to new XTC2 SN 94.
    Return to spares.</pre>



<!-- Date: Thu Feb 28 14:42:16 2008 -->
I swapped two TDCs in the EM timing crate in hopes of generating  more COT-capable spares:  
<bl> 
<li> Slot 5: S/N 163→684  
<li> Slot 7: S/N 609→715  
</bl> <p>  
<!-- Author: jdl -->
<!-- Comment: Thu Feb 28 15;18;49 comment by...Lou DalMonte -->
<pre>TDC 163 Passes Mutrig & RC Testing.
    Grade for XTC -> A    Match XTC1, SN22
    Return to spares.  LD
TDC 609 Passes Mutrig but FAILS RC (Ch53)
    I will look into poss repair of Ch53. LD</pre>

<!-- Comment: Tue Mar 18 15;46;13 comment by...Lou DalMonte -->
<pre>TDC 609 -had CH53 TDC replaced 03/04/08
        -pass all tests 03/18/08
        -grade -> D, match w/XTC2 SN45
        -03/18/08 Return to spares
</pre>


<!-- Date: Tue Mar 18 15:53:38 2008 -->
<pre>TDC 191  (from CDF cage area) 
         - 02/27/08 Failing Ch8 in RC 
         - 02/29/08 Found resistive short N11,6 to N12,28 
         - Repair short affecting Ch8 
         - 03/18/08 Passing all tests. 
           Grade -> A, Match w/XTC1 SN38. Return to spares.</pre>
<!-- Author: Lou DalMonte -->


<!-- Date: Tue Mar 25 15:12:51 2008 -->
<pre> Qty (7) RevF TDC's from detector: 
 TDC830, TDC831, *TDC829, *TDC869, *TDC926, *TDC913 & *TDC852. 
 (*-these already had XTC1 boards, others had no XTC's). 
Verified that all boards pass RC and Mutrig /all tests. 
Verified that all boards also pass XTC Testing. 
The two TDC's (TDC830, TDC831) that did not have XTC's will be matched 
to XTC2 boards, regardless of Grading.(we need RevF with XTC2 spares as per jdl)</pre>
<!-- Author: Lou DalMonte -->
<!-- Comment: Tue Mar 25 15;46;00 comment by...Lou DalMonte -->
<pre>TDC830 Graded -> B, but matched with XTC2 SN45.
TDC831 Graded -> A, but matched with XTC2 SN23.
   Returned ALL, Qty (7) RevF TDC's to spares.</pre>


<!-- Date: Thu May 1 15:34:19 2008 -->
<pre>Two more TDC's added to our good spares inventories 
(from B0 cage, non-working).  
  
ECL, TDC970  
Had  failures relating to Ch 59 and 61. Both TDC's repaired.  
Returned TDC to spares.  
  
LVDS, TDC592  
Had failures relating to anytime one of the following channels were enabled  - >   Ch80, 81, 82,  83.  
Fixed broken net (DATA31_20*).  
Graded as a B, matched with XTC1 SN 233  
  
-Lou </pre>
<!-- Author: Lou DalMonte -->


<!-- Date: Wed Jun 25 15:18:06 2008 -->
<pre>Removed 977 and 984 from CLC00 crate.  We should have done this when we declared CLC02 operational.  They are Rev F ECL boards.</pre>
<!-- Author: jdl -->


<!-- Date: Tue Jul 8 13:57:38 2008 -->
<pre>On request to make RevF w/XTC2 spare: 
TDC869 and XTC2 SN191 are passing all XTC2 tests. 
Move TDC869 to spares..</pre>
<!-- Author: Lou DalMonte -->
<!-- Comment: Tue Jul  8 14;00;45 comment by...Lou DalMonte -->
<pre>Because of need TDC869 was not graded, meaning it could be a possible match with XTC1 if needed in future use.</pre>


<!-- Date: Tue Jul 8 16:19:10 2008 -->
<pre>late entries from July 3rd work: 
 
ECL TDC's 984,977   -> re-configured the pwr fuses, return spares.  
  
ECL TDC 121 from CDF Lab area. Passing all tests, return to spares.  
  
LVDS TDC 514 from 3rd MI shipment,  fixed, return to spares  
  
LVDS TDC 184 from 3rd MI shipment,  fixed, but some evidence of poss prob with loss of VME communication. Gave to Jonathan for long term testing.  
  
LVDS TDC 679 from CDF Lab area. Passing all tests. Gave to Jonathan for long term testing, only due to old tag which stated it was FLAKY. 
  
-Lou</pre>
<!-- Author: Lou DalMonte -->


<!-- Date: Wed Jul 23 16:40:32 2008 -->
TDC 662 returned from Urbana.  It was sent down because of XTC troubles.  I should be tested and mated with a new XTC.
<!-- Author: jdl -->
<!-- Comment: Fri Aug  8 14;13;45 comment by...jdl -->
<pre>"It should be tested..."  Yuck.</pre>


<!-- Comment: Thu Aug 14 11;07;04 comment by...Lou DalMonte -->
<pre>07/31/08  TDC 662 Passes MuTrig and RC. Grade for XTC -> C. Match with XTC2 SN55. Return to Spares. LD</pre>
<!-- Date: Fri Aug 8 14:17:10 2008 -->
<pre>LVDS TDCs moved to cage: 308, 341, 317, 345, 333. 
All are found by Lou to be Flaky 
 
I also moved #393 which is ECL.  It too is flaky.  It had intermittent problems being programmed and caused Global TDC done errors after a few hours of running in the long-term test crate.  After it was removed, test ran smoothly for ~2 days at a time.  </pre>
<!-- Author: jdl -->
<!-- Comment: Thu Aug 14 13;35;00 comment by...Lou DalMonte -->
<pre>Brief description of symptoms:
TDC308 Occasional VME Err w/o known cause.
TDC341 DSP Resets may cause card to be lost. Occasionally, gives VME Err's.
TDC317 DSP Resets may cause card to be lost. Fails RC with all Ch's disabled.
TDC345 Occasional VME Err w/o known cause. Done Timeouts in RC.
Occasional need of FP Reset to revive.
TDC 333 Intermittent. Sometimes FLASH reprogram runs, other times fails. Occasional VME Err w/o known cause. Occasional need of FP Reset to revive.
</pre>


<!-- Date: Thu Aug 14 13:50:33 2008 -->
<pre>TDC Repairs: 
 
TDC 945 ECL 
Failing Ch83. Damaged traces due to previous rework attempts were repaired. TDC's 82, 83 were replaced. Replaced R147, which was found to be of wrong value. This was the cause of the Ch83 problems. Return to Spares. 
 
TDC 309 LVDS 
Failing Ch39, due to 2 open nets ( pins11,38 ). 
Failing Ch15. Replaced a bad TDC. 
Done Timeouts in RC developed. Found the open net LD(13). This caused a header word mismatch and the Done Timeouts. 
Return to spares. </pre>
<!-- Author: Lou DalMonte -->


<!-- Date: Tue Aug 19 17:17:42 2008 -->
TDC 377 was giving done timeouts on every event in slot 11 of cmu00.  I swapped it out and installed TDC 369.
<!-- Author: jdl -->
<!-- Comment: Tue Sep  2 14;25;59 comment by...Lou DalMonte -->
<pre>Prob appears to be that the FIFO's are  being held in reset. TDC_FIFO_RESET bit in Contol Reg cannot be set to LO. Will investigate further.</pre>
<!-- Comment: Wed Sep  3 15;05;58 comment by...Lou DalMonte -->
<pre>09/03/08 TDC 377. Problem was found and repaired. Found an open net 019,16 -> Q16,33. Net-name (TDC_FIFO_RESET). This broken connect was causing FIFO's to be held in RESET always.
Board passing all tests. Return to Spares. LD</pre>
<!-- Date: Fri Aug 22 10:29:05 2008 -->
<pre>TDC 144 ECL 
(a board from shipment4 MI) 
No Problems found Mutrig or RC. 
Repeated testing dates 08/14, 08/20, 08/22. ALL OK. 
Return to spares.</pre>
<!-- Author: Lou DalMonte -->


<!-- Date: Tue Sep 2 14:18:01 2008 -->
<pre>TDC 265 ECL Rev D 
From shipment #3 MI. 
Repaired 4 open nets at TDC's 18 & 17. 
Replaced a bad TDC14. 
Repaired 1 open net at PQ12, preventing FIFO read clocks. 
OK - Return to Spares. 
</pre>
<!-- Author: Lou DalMonte -->



<!-- Date: Tue Sep 2 14:53:27 2008 -->
<pre>TDC 926 LVDS RevF 
08/22 return from Field. (COT16, Slot 17) 
Symptom reported: Frequent Done Timeouts. 
09/22/08 Passing all tests in Teststand. Not able to reproduce. 
Will run overnight.</pre>
<!-- Author: Lou DalMonte -->
<!-- Comment: Wed Sep  3 15;01;33 comment by...Lou DalMonte -->
<pre>09/23/08 Testing still all OK next day. I will give this TDC to Jonathan for longer term testing, but I believe this card is OK.</pre>



<!-- Comment: Thu Nov 13 15;24;09 comment by...Lou DalMonte -->
<pre>11/13/08 This board still passing all tests in TDC teststand.</pre>
<!-- Date: Wed Oct 8 12:03:35 2008 -->
<ul>  
<li>Swapped TDC in b0cot02 slot 3 per request 436 for XTC problems (removed TDC #700 and XTC2 #93)  
<li>Interchanged TDCs in slots 13 and 14 of b0cot08 to try to diagnose longstanding L2 XFT problem  
<li>Needed to reseat slot 7 in b0cot01. It didn't come back when the wall power came back on. If it fails again on Friday, we should replace it. 
</ul>
<!-- Author: jdl -->
<!-- Comment: Wed Oct  8 12;08;04 comment by...jdl -->
<pre>Eiko noticed that the board that moved from slot 14 to 13 is one of the boards with a timing offset.  T0 is ~8ns less than a typical card.</pre>

<!-- Comment: Wed Oct  8 13;02;54 comment by...Heather -->
<pre>In b0cot02 slot3, now we have TDC #721 and XTC2 #94. (to help me if I need to remember about this).  TDC #700 and XTC2 #93 was removed because it was causing problems both at L1 and L2 of the finder.  Also, the XTC BC was off by 1.</pre>

<!-- Comment: Thu Oct 16 17;18;09 comment by...jdl -->
The XTC2 problem in cot08 followed the card.  Eric swapped it earlier this week.  The problem TDC was #519 with XTC2 #166.  The P3 connector on the TDC was pretty mangled.  That may explain the XTC issue.  We should fix that and return the board to service after retesting.

<!-- Comment: Thu Oct 16 17;19;28 comment by...jdl, again -->
<pre>Only the shield is mangled.  The body of the connector is OK.</pre>


<!-- Comment: Fri Oct 24 14;36;35 comment by...Lou DalMonte -->
<pre>TDC 519 : The mangled P3  connector shield was replaced. LD
Passing MuTrig and RC, all tests OK.
XTC2 testing fails with xtc2 SN 166 on test 17.
Tried with new xtc2 SN 195,  gives same results. I will need to re-grade for XTC. Maybe this is a grade F board..</pre>
<!-- Comment: Fri Oct 24 15;07;47 comment by...Lou DalMonte -->
<pre>More on TDC 519
Grading TDC 519 GRADING results  - &gt;  GRADE F 
(fails immediatly the TDC Quick Test with all XTC1 test  boards) </pre>
<!-- Comment: Fri Oct 24 15;29;28 comment by...Lou DalMonte -->
<pre>TDC 519 - Return to  spares (LVDS RevC / no XTC)</pre>
<!-- Date: Tue Oct 21 14:45:28 2008 -->
<pre>TDC 145 ECL Rev D  
From shipment #4 MI.  
Repaired 3 open nets. LD(26), LD(22) & CLK_6 between TDC53 and TDC54. 
NOW OK - Return to Spares. </pre>
<!-- Author: Lou DalMonte -->





<!-- Date: Tue Nov 11 14:35:00 2008 -->
<pre>On Friday we swapped out TDC 542 from b0cot01 slot 7.  It would not pass config after the power outage.  We tried power cycling the crate, reseating the TDC and reloading the FRAMs.  The error was that it was an incorrect board type.  The new board in that location is #662</pre>
<!-- Author: jdl -->
<!-- Comment: Wed Nov 26 13;56;36 comment by...Lou DalMonte -->
<pre>Ron Moore and myself, we looked at this TDC at the teststnd for a couple of hours. No consistent fault condition was present. It appears this TDC is very erratic and fails in several different ways, while the DSP is enabled. No good conclusions made to pursue a diagnosis at this time.</pre>


<!-- Date: Wed Nov 26 16:01:10 2008 -->
<pre>TDC  Grading - 3 boards graded for XFT. 
TDC519 -&gt; Grade B, TDC514 -&gt; Grade B, TDC309 -&gt; Grade B. 
We need RevC TDC XTC2 spares so we will match TDC519 w/ xtc2 SN195. 
TDC519 passes all XTC2 tests. Move TDC519 to spares.</pre>
<!-- Author: Lou DalMonte -->


<!-- Date: Thu Dec 18 11:54:22 2008 -->
<pre>TDC 696 was tested in Urbana with several XTCs and every XTC failed simple R/W and connectivity tests.   The TDC is currently in the possession of JDL.</pre>
<!-- Author: KPitts -->