<!-- Title: 15:39:42  Mon May 17 2004 -->
<!-- Crewchief: Rong-Shyang -->
<!-- Op1: Jared (Detector) -->
<!-- Op2: Reid (DAQ) -->
<!-- Op3: Ben (Monitoring) -->
<!-- Op4: Marcel (shadowing) -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>  </pre>


<!-- Date: Mon May 17 15:56:56 2004 -->
<h3> Supervised access report </h3> 
Access by Rong-Shyang, Eric and Marcel with David and Stan in the control room. <br>
 <br>
Light level measurement for  
e382 bus7 low and e152 bus5 low problem. <br>
 <br>
Both buses have normal light out with <br>
e382-bus7 -> 290 uW <br>
e152-bus5 -> 544 uW <br>
 <br>
Also e382 doesn't have error since run 182268, Apr29, 2004. <br>
 <br>
For e152, although we see bus 5 on scope, but the DAQ keeps getting bus5 low error. So we decided to swap doim to see if it's RX/DAQ problem. 
 <br>
<li> swap e152 with e153, error change to pipe 8 
<li> swap e151 with e152, error change to pipe 6 
<li> swap e141 with e152, error change to pipe 1 
 <br>
It's the internal problem, but we can't think of the reason. 
 <br>
<li>We also check the possible phase shift on this bus, but no 
clear difference can be seen.
<!-- Author: Rong-Shyang -->


<!-- Date: Mon May 17 16:04:14 2004 -->
<center><a href='/elog/notebooks/silicon/Mon_May_17_2004_160330.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4101' ></a><a href='/elog/notebooks/silicon/Mon_May_17_2004_160352.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4103' ></a></center>
<!-- Author: Rong-Shyang -->
<!-- Comment: Mon May 17 16;06;16 comment by...Rong-Shyang -->
<pre>Light measurement on e152 bus7(ch2) and bus2(ch1).
Bus 7 has 290 uW and has no error recently.
Some other thing is flaky?</pre>


<!-- Date: Mon May 17 16:07:16 2004 -->
<center><a href='/elog/notebooks/silicon/Mon_May_17_2004_160643.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4105' ></a><a href='/elog/notebooks/silicon/Mon_May_17_2004_160657.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4107' ></a></center>
<!-- Author: Rong-Shyang -->
<!-- Comment: Mon May 17 16;09;18 comment by...Rong-Shyang -->
<pre>Light measurement of e152.
Left plot : bus5(ch2) and OBDV(ch1). Bus5 has 544uW which should be enough to be identified by RX.

right plot : look at possible phase shift between buses.
bus5(ch2) and bus4(ch1) do not have much difference in timing.
</pre>

<!-- Comment: Mon May 17 17;09;19 comment by...Rong-Shyang -->
<pre>Suen suspect the duty cycle is small, and we forgot to measurement on that. One idea is to get a wide DutyCycle RX (green one) and try to match it.</pre>
<!-- Date: Mon May 17 17:07:57 2004 -->
Entered in Silicon database: <ul>
<li>Ladder: SB1W5L2 
<li>problem n: 63 
<li>Status: needs_access_2hr
<li>Type: Optical_other
<li>Action: Diagnosis_update
<li>entered by: rong 
<li>description is: Light level measurement again<a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=196&frame=2&anchor=16:07:16'> see here.</a> Could try on swap different RX or FTM board. 
<li>See <a href="http://www-cdfonline.fnal.gov/internal/usr/lmiller/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B1W5&layer=L2">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Mon May 17 17:12:45 2004 -->
<p><font size=+1> D-mode Calibration Job Finished </font>
<p><pre>
This is a computer-generated silicon D-mode calibration status report.
Silicon D-mode calibration job 340 FAILED.

Job info
--------
Calibration type : standard
DPS-off run      : 182721
DPS-on run       : 182720
Requested by     : Olga
Submitted        : 2004-05-17 10:37:38
Completed        : 2004-05-17 17:12:22
Status           : FAILURE

Last processing stage reported : starting histogram transfer
Failure diagnostics :
You have no controlling tty and no DISPLAY.  Cannot read passphrase.
lost connection
</pre>
<!-- Author: D-mode script run_calib_analysis.tcl -->
<!-- Comment: Tue May 18 08;11;33 comment by...Jason -->
This is different, but related to the problems before.
<BR>
I have intervened manually to finish this calibration and tweaked the code to keep it from happening again.
<!-- Date: Mon May 17 17:32:17 2004 -->
<pre>performed LLM SB2W6L2/e262.</pre>
<!-- Author: ssf, erik -->


<!-- Date: Tue May 18 00:43:35 2004 -->
<h3>AAK1 is giving trouble again. </h3>
 <a href=http://www-bd.fnal.gov/cgi-mcr/elog.pl?nb=2004&action=view&page=374&anchor=224715&hilite=22:47:15- target=_top> MCR elog </a>
 
There seems to be some rad on BLMs. <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=338&frame=2&anchor=00:10:08.281'> 
See here </a> and <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=338&frame=2&anchor=00:37:49.391'> 
here </a>
<!-- Author: Rong-Shyang -->


<!-- Date: Tue May 18 03:37:21 2004 -->
<pre>Store 3506 is bad. 
LostP shot up to 100K for an hour. CDF no data yet for 3 and half hour. 
The plan is waiting for beam stable and run the CDF without Silicon for an hour then bring Silicon in.</pre>
<!-- Author: Rong-Shyang -->
<!-- Comment: Tue May 18 03;39;03 comment by...Rong-Shyang -->

<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=338&frame=2&anchor=01:12:12.350'>
Plots </a> in shift elog.

<!-- Comment: Tue May 18 05;08;11 comment by...Rong-Shyang -->
<pre>After running CDF without silicon for an hour and the LostP is 
stably decreasing, I gave the permision to have silicon in the DAQ when the LostP rate is at 20.5KHz in 6 mins average rate.
</pre>


<!-- Date: Tue May 18 05:44:51 2004 -->
<table><tr><td><a href=/elog/notebooks/silicon/Tue_May_18_2004_054322.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4109  align='top'></a></td><td>SVT spymon plot</td></tr></table>
<!-- Author: Rong-Shyang -->


<!-- Date: Tue May 18 05:58:41 2004 -->
Tevatron quench at 5:50, 40 minutes after silicon is in. 
ACEs had Si went though cold start with no problem. 
Also no clear rad dose has been seen.<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=338&frame=2&anchor=05:50:42.021'> 
here </a> 
<!-- Author: Rong-Shyang -->


<!-- Date: Tue May 18 08:42:44 2004 -->
Entered in Silicon database: <ul>
<li>Ladder: SB3W8L2 
<li>problem n: 1191 
<li>Status: needs_observation
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: rong 
<li>description is: Light level measurement on bit7 looks fine, see <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=196&frame=2&anchor=15:56:56'> here</a>. Also there is no error since run 182268, Apr29, 2004. Keep watching. 
<li>See <a href="http://www-cdfonline.fnal.gov/internal/usr/lmiller/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B3W8&layer=L2">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue May 18 14:04:01 2004 -->
Entered in Silicon database: <ul>
<li>Ladder: SB5W0L3 
<li>problem n: 1099 
<li>Status: needs_observation
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: rong 
<li>description is: No error since run 182463, May07, 2004. Change to Monitoring group and keep watching it. 
<li>See <a href="http://www-cdfonline.fnal.gov/internal/usr/lmiller/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B5W0&layer=L3">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Tue May 18 20:14:38 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4110&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4111  width='300' align='top'></a></td><td>SPYmon plot for store 3510</td></tr></table>
<!-- Author: Rong-Shyang, Marcel -->


<!-- Date: Tue May 18 22:10:54 2004 -->
Problem powering wedge sb0w4 after <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=340&frame=2&anchor=22:04:40.240'> 
hockerization.</a>  Shows how the aces need to review the procedure.
<!-- Author: Jared -->


<!-- Date: Tue May 18 22:47:03 2004 -->
 
<li>f314 yellow or red on SVXMon, see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=340&frame=2&anchor=20:58:29.167'> 
here </a> . 
<li>e284 phi side chip failure (red) on svxmon; 
<li>e463 red on chip status  
 
we may find some time to revisit them.  
<!-- Author: Rong-Shyang -->
<!-- Comment: Wed May 19 02;51;17 comment by...rainer -->
<pre>some of these are known ,see C0 reference folder.</pre>


<!-- Comment: Wed May 19 05;47;34 comment by...Rong-Shyang -->
<pre>Yes, they are showing in the reference plots.
I am just wondering what's the reason cause the failure on them and maybe we can look at them again and see possible fixing.</pre>
<!-- Date: Wed May 19 05:38:48 2004 -->
<li>B0PAGC lose rate shot up to 90KHz due to possible RF glitch. see 
<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=341&frame=2&anchor=04:40:04.957'> 
shift elog </a> . 
Silicon was brought to Standby due to TeVmon error state. <br> 
<li>I was paged after SciCo's talking with MCR. although MCR still saying that they are investigaing and it shouldn't be happened again, I have to give permission for them to run silicon after seeing B0PAGC back to the level it was.<br> 
<li>DAQ ace was concerning about several error messages from VRB_SVX_02 and SRC, see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=341&frame=2&anchor=04:39:52.542'> 
here </a>, when they had a few run transitions due to DAQ error and bad beam-condition. So I reset the SRC via Svxdaq first in case they got more trouble later. I hope this is fine. 
  
<!-- Author: Rong-Shyang -->



<!-- Date: Wed May 19 11:23:49 2004 -->
<center><a href='/elog/notebooks/silicon/Wed_May_19_2004_112259.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4113' ></a><a href='/elog/notebooks/silicon/Wed_May_19_2004_112313.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4115' ></a></center>
<!-- Author: SCN -->
<!-- Comment: Wed May 19 11;56;01 comment by...SCN, Marcel, Lester, Sebastian -->
Moonlighting for SVT...
<h3>Back End State Bits</h3>
SVT wants to study timing, so we verified that the BE state bits the SRC passes to the FIB in fact work.<ul>
<li> Left:  A L1A arrives between a L1A (<i>L1A1</i>) and a L1DONE (<i>L1D1</i>) meaning the SRC believes the Chips are in DIGITIZE.  The Back end state bits (<i>FBESTA</i>) == 1
<li> Right: A L1A arrives after the L1DONE but before READBUSY (<i>RDBSY1</i>) goes low, meaning the SRC says the Chips are in READOUT.  The Back end state bits (<i>FBESTA</i>) == 2
</ul>
We did not check to see that these bits are never set when they shouldn't be, but it is reasonable to think they are working.


<!-- Date: Wed May 19 15:25:40 2004 -->
<table><tr><td><a href=/elog/notebooks/silicon/Wed_May_19_2004_152529.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4119  align='top'></a></td><td>SVT spymon plot for run 182843 for same store 3510</td></tr></table>
<!-- Author: Rong-Shyang -->


<!-- Date: Wed May 19 20:17:57 2004 -->
Tevatron lost the beam due to T:VE16 tripping off. see <a href=http://www-bd.fnal.gov/cgi-mcr/elog.pl?nb=2004&action=view&page=380&anchor=194012&hilite=19:40:12- target=_top> MCRelog </a>. Post-quench checkout ok by ACEs. 
  
<!-- Author: Rong-Shyang -->


<!-- Date: Wed May 19 20:38:57 2004 -->
<h3> IMON Blues</h3> 
We just saw, that Imon silently died without giving any messages. 
It stopped updating May 17th, 09:35.  
The error messages was Java.lang.OutofMemoryError. The gui was still reacting to mouse clicks, but did not update anything. 
 
funny ....
<!-- Author: Rong-Shyang,Marcel -->


<!-- Date: Wed May 19 21:53:18 2004 -->
We got an hour quiet time to try on 2 ladders. 
<li> f111 was out for a while. After power it up, we got AVDD 600mA and DVDD 290mA. It failed INIT and no data back. Tried increase Rdrivers to all chips; tried to by pass 4 chips (1st hybrid phi side). Still had no data back. 
 
<li>e034 has stuck cell ID on chip2(8b). Tried to change ISEL bits to increase FE current but still got cellID out of sync. But the cell ID didn't stuck, it just out of sync in one event but still keep rotation of Caps like others. 
  
<!-- Author: Rong-Shyang, Marcel -->

<!-- Date: Wed May 19 21:55:45 2004 -->
Entered in Silicon database: <ul>
<li>Ladder: IB1W1L1 
<li>problem n: 992 
<li>Status: needs_work
<li>Type: Unknown
<li>Action: Diagnosis_update
<li>entered by: rong 
<li>description is: Tried again. Still no data back. see <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=196&frame=2&anchor=21:53:18'> elog</a>.    
<li>See <a href="http://www-cdfonline.fnal.gov/internal/usr/lmiller/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B1W1&layer=L1">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Wed May 19 21:58:55 2004 -->
Entered in Silicon database: <ul>
<li>Ladder: SB0W3L4 
<li>problem n: 1053 
<li>Status: needs_work
<li>Type: Other
<li>Action: Action_request
<li>entered by: rong 
<li>description is: Tried to increase FE current but still got CapID out of sync for chip 2, see <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=196&frame=2&anchor=21:53:18'> elog </a> 
<li>See <a href="http://www-cdfonline.fnal.gov/internal/usr/lmiller/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B0W3&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Thu May 20 00:25:20 2004 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4122&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4123' ></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4120&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4121' ></a></center>
<!-- Author: Rong-Shyang, Marcel -->
<!-- Comment: Thu May 20 00;25;48 comment by...Rong-Shyang, Marcel -->
<pre>IMon pinky on ISLB4W0 5VDOIM.
It turns out this current drop since two days ago, but due to problem on IMON, no one notice until now.
It dropp from 1650 down to 1540.</pre>

<!-- Comment: Thu May 20 00;30;02 comment by...Rong-Shyang, Marcel -->
<pre>the ladders in this barrel show no anomalies in the bitreporter for the last 30 days. We 'll have a look at it tomorrow

</pre>


<!-- Comment: Thu May 20 00;30;43 comment by...Rong-Shyang, Marcel -->
<pre>meaning with the data from this night
</pre>
<!-- Date: Thu May 20 00:30:18 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4124&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4125  width='300' align='top'></a></td><td>SVT spymon plot for store 3513, run182873</td></tr></table>
<!-- Author: Rong-Shyang, Marcel -->



<!-- Date: Thu May 20 07:46:56 2004 -->
Tevatron lost beam again for store 3513 due to PS trip. see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=344&frame=2&anchor=05:48:44.382'> 
shift-elog </a> and <a href=http://www-bd.fnal.gov/cgi-mcr/elog.pl?nb=2004&action=view&page=381&anchor=054954&hilite=05:49:54- target=_top> MCR-elog </a>. 
 
No rad. dose seen, and coldstart ok by ACE.
<!-- Author: Rong-Shyang -->


<!-- Date: Thu May 20 08:23:40 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4126&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4127  width='300' align='top'></a></td><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4128&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4129  width='300' align='top'></td></tr></table><br>
Zoom in on I 5Vdoim change. It steped down at end of the day 1233. (May 18 2004 ?)
<!-- Author: Rong-Shyang -->


<!-- Date: Thu May 20 09:57:54 2004 -->
IB4W1L1 is reported to have longest RO time by David Stuart. 
This guy has half of HDI (1hybrid) unbiased, <a href="http://www-cdfonline.fnal.gov/internal/usr/lmiller/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B4W1&layer=L1"> problem DB </a>, and the noise leads to fire almost all the channles for the r-phi side chips. 
If we can't fixed the bias problem, perhaps we should sparse these chips out. 
 
<pre> 
--------------- 
one event in run 182839 
 
1043100 a024 7f08 c1c1 f411 6dfc 8f24 00f3 03f3 <- f411 start; chip8f 
1043120 0409 0507 060e 0709 0808 0b07 0c0f 0d08 <- has 123 channels RO 
1043140 0e0e 0f13 1010 110f 1213 1309 1416 1504 
1043160 1618 170f 180f 190f 1a0f 1b1b 1c0e 1d1c 
1043200 1e14 1f09 201a 2117 2219 231c 240e 2516 
1043220 261c 271a 2820 2917 2a1a 2b1e 2c11 2d1b 
1043240 2e1d 2f18 3017 311a 3214 331a 341f 351a 
1043260 361a 3716 3818 3920 3a1c 3b18 3c14 3d0d 
1043300 3e29 3f1d 4016 411d 42f3 4336 44f3 4510 
1043320 463f 47f3 483c 49f3 4a30 4bf3 4c32 4df3 
1043340 4e3c 4ff3 5016 513c 52f3 5338 54f3 5536 
1043360 560f 57f3 5848 59f3 5a13 5b32 5cf3 5d4f 
1043400 5ef3 5f32 60f3 6144 62f3 634e 64f3 65f3 
1043420 663a 67f3 6834 69f3 6a42 6b01 6c06 6d3e 
1043440 6ef3 6f3b 70f3 714d 72f3 733c 74f3 7543 
1043460 76f3 7741 78f3 7934 7af3 7b40 7cf3 7d3f 
1043500 7ef3 7f38 8e24 0035 0131 022e 0332 0438 <- chip 8e has 128ch RO 
1043520 0531 0638 0733 0839 093b 0a33 0b2e 0c3e 
1043540 0d30 0e32 0f37 1037 1130 125a 13f3 1466 
1043560 15f3 1661 17f3 1878 1904 1a5f 1bf3 1c72 
1043600 1df3 1e82 1ff3 2071 21f3 2268 23f3 247b 
1043620 25f3 2664 27f3 2861 29f3 2a61 2bf3 2c6e 
1043640 2df3 2e73 2ff3 306a 31f3 326e 33f3 345c 
1043660 35f3 3678 37f3 3873 39f3 3a5d 3bf3 3c68 
1043700 3d03 3e6c 3f09 406d 4103 4277 4329 4403 
1043720 456c 46f3 4771 48f3 496a 4af3 4b78 4cf3 
1043740 4d6d 4ef3 4f63 50f3 5176 52f3 5361 54f3 
1043760 5576 56f3 576a 5803 5962 5a06 5b6b 5cf3 
1044000 5d6e 5e1c 5ff3 6062 61f3 6276 63f3 6466 
1044020 65f3 6673 6711 685c 69f3 6a74 6bf3 6c6a 
1044040 6d0c 6e30 6f2d 702c 7132 7232 7331 742a 
1044060 7533 762e 7731 782b 7931 7a34 7b2e 7c34 
1044100 7d36 7e2a 7f30 8d24 00f3 1502 1609 17f3 <- chip 8d,  7ch RO 
1044120 27f3 2809 29f3 7f08 8c24 002a 012b 0228 <- chip 8c,  129 ch RO 
1044140 0304 046d 050c 066a 07f3 0865 09f3 0a60 
1044160 0bf3 0c74 0df3 0e62 0ff3 106e 11f3 1260 
1044200 13f3 1476 15f3 1660 17f3 186b 19f3 1a6e 
1044220 1bf3 1c70 1df3 1e6c 1ff3 2061 21f3 226a 
1044240 2307 2475 25f3 2661 2706 2863 29f3 2a6b 
1044260 2b09 2c68 2df3 2e73 2f07 3075 3109 3271 
1044300 33f3 3478 3505 366e 3708 3827 3927 3a33 
1044320 3b31 3c2f 3d2f 3e31 3f31 4032 4130 422c 
1044340 4330 4427 4528 462a 472c 4829 4969 4af3 
1044360 4b67 4c10 4d59 4ef3 4f5c 50f3 516f 5210 
1044400 536b 54f3 5563 560a 576d 58f3 5967 5a02 
1044420 5b60 5cf3 5d6d 5ef3 5f5c 60f3 6174 6205 
1044440 6367 64f3 6518 6666 670e 6856 6908 6a71 
1044460 6b07 6c64 6df3 6e78 6f08 7058 71f3 7269 
1044500 73f3 7471 75f3 7670 77f3 786c 79f3 7a66 
1044520 7b08 7c78 7df3 7e5d 7f01 9b24 9a24 7f01 <- chip 9b, 9a 
1044540 9924 000c 0102 9824 b724 b624 b524 6105 <- chip 99, 90, b7, b6, b5 
1044560 620d 63f0 64f3 b424 a324 30f3 3109 3203 <- chip b4; a3 with 15chRO  
1044600 3c03 3d0c 3e08 6406 650a 66f3 7204 7309 
1044620 74f3 7a03 7b0c 7cf3 a224 42f3 430a 4404 <- chip a2, 9 ch RO 
1044640 6ef3 6f09 7006 7df3 7e0a 7f0a a124 00f3 <- chip a1, 11 ch RO 
1044660 0606 070c 08f3 0909 0a07 1b03 1c09 1df3 
1044700 2cf3 2d0b 2e01 a024 0503 0609 0704 6e02 <- chip a0, 15 ch RO 
1044720 6f0a 7001 7704 780b 7908 7a0a 7b09 7c05 
1044740 7d09 7e0d 7f0c c1c1 0000 f412 6dfc 8f24 <- f412 total 90 Ch RO 
1044760 0009 01f3 8e24 8d24 8c24 5df3 5e09 5ff3 
1045000 9b24 9a24 9924 1ff3 2009 21f3 9824 43f3 
1045020 440b 45f3 6a03 6b09 6c04 6d0a 6e06 6f0b 
1045040 7008 7406 750a 7609 7708 7808 790a 7a0a 
1045060 7b0a 7c08 7d0b 7e0b 7f07 b724 06f3 0710 
1045100 08f3 b624 7d07 7e0d 7f07 b524 b424 2202 
1045120 230c 24f3 3c06 3df0 3e08 7106 720a 7309 
1045140 7409 7509 760c 7709 780a 790d 7a09 7b09 
1045160 7c0e 7d0f 7e0e 7f0f a324 00f3 3001 3109 
1045200 32f3 a224 09f3 0a0a 0bf3 a124 79f3 7a0b 
1045220 7b03 a024 0ef3 0f09 1001 1f05 200f 21f3 
1045240 7f07 c1c1 0000 f413 6dfc 8f24 00f3 20f3 <- f413 total 127 ch RO 
 
</pre>
<!-- Author: Rong-Shyang -->

<!-- Date: Thu May 20 10:01:07 2004 -->
Entered in Silicon database: <ul>
<li>Ladder: IB4W1L1 
<li>problem n: 1250 
<li>Status: needs_quiet_time
<li>Type: BIAS
<li>Action: Diagnosis_update
<li>entered by: rong 
<li>description is: The unbiased sensor seems to fire almost all the channels RO in rphi chips.<a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=196&frame=2&anchor=09:57:54'> see elog</a>. If can't fixed problem, sparse those chips out since no physics is seen. 
<li>See <a href="http://www-cdfonline.fnal.gov/internal/usr/lmiller/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B4W1&layer=L1">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Thu May 20 14:25:44 2004 -->
Store 3516. 
Two hours after Shot-setup completed, CDF started to take data without Silicon. LostP and B0PAGC rates are jumping around. 
see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=345&frame=2&anchor=14:23:17.718'> 
plots</a> in elog.  
<!-- Author: Rong-Shyang -->
<!-- Comment: Thu May 20 16;47;12 comment by...Rong-Shyang -->
Due to a diconnected damper, MCR couldn't have good control nor pull down the abort gap lost rate. CDF keep taking data without silicon for the rest of the store.
 <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=346&frame=2&anchor=16:35:51.952'>
shift elog </a> 


<!-- Date: Thu May 20 23:44:53 2004 -->
b0fib07 is red in the VxWorks System Monitor while CDF is taking data without silicon. ACE tried vxlogin but failed. see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=346&frame=2&anchor=23:19:07.836'> 
elog.</a> 
I also tried vxlogin but failed. b0fib07 is not pinable. 
minicom has access but no response when I try 'reboot'. 
 
Then I did the hardreboot way via RC and did shperded crate 
and it worked. Also got activities and messanges from my minicom window. 

<!-- Author: Rong-Shyang -->


<!-- Date: Fri May 21 02:01:05 2004 -->
<pre>Software protection trip on DVDD at 01:45 of   
e000, e003, e080, e083, e240, e3b0, e500, e501, and e502. 
Ask ACE to bring them back to standby and config them without 
problem.
</pre>
<!-- Author: Rong-Shyang -->
<!-- Comment: Fri May 21 02;39;52 comment by...Rong-Shyang -->
They tripped together with BMU trip on NE24, see <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=347&frame=2&anchor=02:10:09.989'>
elog</a> .


<!-- Date: Fri May 21 07:12:59 2004 -->
Tevtron quench, with small rad does seen at 4 BLMs, see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=347&frame=2&anchor=07:05:16.929'> 
elog </a> and <a href=http://www-bd.fnal.gov/cgi-mcr/elog.pl?nb=2004&action=view&page=384&anchor=065856&hilite=06:58:56- target=_top> MCR elog </a>. But no entry could be found for SVX BLMs summary plot database. Also Tevatron did lost some proton bunches an hour earlier, see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=347&frame=2&anchor=06:06:41.273'> 
elog </a>
<!-- Author: Rong-Shyang -->
<!-- Comment: Fri May 21 07;18;47 comment by...rong-shyang -->
<pre>ACEs did post-quench checkout and brought Silicon through Coldstart without problem.</pre>


<!-- Date: Fri May 21 11:11:16 2004 -->
<table border=0><td><tr><td bgcolor=#e0e030><H1> Crowbars Blown </h1> 
<pre> 
We now have ADCMon data from a run (182839) taken on Wednesday with silicon and I have checked L00 for physics.  It looks like many ladders are unbiased.  The complete list is: 
 
HDI        chips which look unbiased 
------------------------------------ 
f803        0,1,2 
f813        0,1,2 
f833        0,1,2 
f841        0,1 
f843        1,2 
fa03        0,1,2 
fa13        0,1,2 
fa21        0,1 
fa22        0,1 
fa23        0,1,2 
fa43        1,2    - Chip 1 has large hump in ADC ~150-225 
fa50        0,1 
fa51        0,1 
fa52        0,1 
fa53        0,1,2  (one of these is known to be unbiased since ~1 year ago) 
 
 
Note that that this time not only narrows blew, and in fact the entire wedge 5 seems to have blown.  In the last run processed before the quench of store 3494 (182588 taken last Thursday 5/13), all look fine.  By my count, a total of 22 bias lines were affected. 
 
 
 
</pre> 
</td></tr></td></table>
<!-- Author: bbrau -->
<!-- Comment: Mon May 24 15;58;24 comment by...reid -->
CAEN info for the blown crowbars:
<table border="1">
<tr>
<th>ladder</th>
<th>crate</th>
<th>slot</th>
</tr>

<tr>
<td>f803</td>
<td>7</td>
<td>5</td>
</tr>

<tr>
<td>f813</td>
<td>8</td>
<td>9</td>
</tr>

<tr>
<td>f833</td>
<td>4</td>
<td>7</td>
</tr>

<tr>
<td>f841,f843</td
<td>8</td>
<td>1</td>
</tr>

<tr>
<td>fa03</td>
<td>15</td>
<td>9</td>
</tr>

<tr>
<td>fa13</td>
<td>16</td>
<td>1</td>
</tr>

<tr>
<td>fa21,fa22, fa23</td>
<td>12</td>
<td>9</td>
</tr>

<tr>
<td>fa43</td>
<td>15</td>
<td>7</td>
</tr>

<tr>
<td>fa50,fa51, fa52, fa53</td>
<td>15</td>
<td>1</td>
</tr>
</table>
<!-- Date: Fri May 21 11:23:38 2004 -->
<h3> Power supply gui problems </h3> 
After some problems with the online file servers, the power supply gui froze. We tried to bring it back, but no had no success (timeing out) 
We then tried to reboot svxinter which also failed, as the crate cpu couldn't load the startup files. After consulting with Steve  N. and checking that there is no beam for a long time (at the time of the failure, the Silicon was on for cosmics), we decided to leave it as it is and wait for the file servers to come back. 
After 15 minutes, they came back and we recovered svxinter and the Power supply gui. 
No ladder tripped during this 
 
 
  
<!-- Author: Rong-Shyang, Marcel -->


<!-- Date: Fri May 21 15:57:31 2004 -->
<h2> Cooling Water  Trip</h2> 
I was paged, as they had a cooling water trip. the interlocks killed the SVX/ISL/L00 (see<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=348&frame=2&anchor=15:41:44.828'> 
main elog </a>). I flipped the 5V kill switches ... 
Silicon Cooling has been paged as well 
 
   
<!-- Author: Marcel -->
<!-- Comment: Fri May 21 16;11;29 comment by...Marcel -->
<pre>for the record, the Silicon was shut down at 15:40</pre>


<!-- Date: Fri May 21 16:37:46 2004 -->
<h2> Crates are going down</h2> 
both the FIB and VRB crates are shut down due to the cooling water  problems  
<!-- Author: Marcel -->


<!-- Date: Sat May 22 02:12:58 2004 -->
<h2> SVXII cooling water fest</h2> 
<ul> 
<li>Cooling Water came back and Andy recovered the SVX cooling. </li> 
<li>We powered all the crates, hockerized the caens, reset svxinter, rebooted b0dap42-44 and brought up  the ps gui.</li> 
<li>Then we checked the clock and clock fanout in the first floor.</li> 
<li>We reset all the crate CPU's in sequence in order to reload the VRB's </li> 
<li>After checking the system with svxdaq and some glink synch blues with b0fib02(it came back by softly touching the cables), we ran with guinea pigs and svxdaq. There we had some problems with running 2 HDI' as we did only get data from one HDI. 
This might be related to another GLINK blues. 
We checked all GLINKs again with svxdaq.</li> 
<li>Step by Step we switched on the silicon and put it through cold start config.</li> 
<li>We had some trouble with the fib crates and rebooted all of them.</li> 
<li>On Activate we got L1 done timeout and Resonance errors. e133 was outed as being the bad guy by the vrb resfinder. however disabling this ladder didn't fix the problem. We suspected some noise issues on vrb daisy chain, so we brought back the VRB crates in order 02 (no power cycle, only VRB reset),00,01,03,04,05 but that did not help. There was no VRB giving READbusy or SCANbusy, so we don't understand the source of the L1 done timeouts. We also never got a !DONE timeout. We checked the ignore VRB flag 28002080 was 0 , so we ruled this out. Then we checked Ghostbuster settings and changed the kicks to 0xa and we were able to run coldstart. We realized that the number has been set to 8 recently, so we set it back to 10 and ran without problems in the cold start config. </li> 
<li> we took a test run 183007 and logged it via the software event builder (Hardware EVB not yet available)</li> 
<li> the sili01,02 were also shutdown by the power cut, but only sili02 came back gracefully, sili01 is stuck with a disk check problem on /data2, which requires root access) </li> 
<li> Two pinkies appeared, f432 has avddd is 1075 and it should be between 1010 and 1030. e053 recovered. </li> 
</ul>  
<!-- Author: Andy,Rainer, Rong-Shyang,Marcel -->
<!-- Comment: Sat May 22 08;43;09 comment by...william -->
<pre>At about midnight the 'other' SPL asked why we had
changed from 11 kicks to 8 in the ghostbuster ...</pre>
<!-- Comment: Sat May 22 09;31;59 comment by...rainer -->
this because the are running now >22 kHz. There is no mechanism as far as I know that produces L1DONE timeout because of a resonance condition - at least I have seen these errors in conjunction with a resonance condition never before. My suspicion is that relaxing the # of kicks back to 10 only cures a sympton, not the cause. we have been runnning in the coldstart configuration without any problems earlier. <p>
also it was 10 kicks before, btw.
<!-- Comment: Sat May 22 09;44;03 comment by...rainer -->
See silicon elog entry<a href=
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=193&frame=2&anchor=17:59:55'>
here </a>
<!-- Date: Sat May 22 08:20:57 2004 -->
<pre>There is alarm for 1RR21F_1 again, when I ran down to check, I saw the upper crate modules were off (no led light for b0svx00 and VRBs) but the PS for upper crate was on. After a few minutes, the power was back by itself and we could read back voltages in iFix. Rob brought a voltmeter down, and the voltage measured on crate was correct, eg. +5V, +-12V. Anyway, Rob suggested we try to replace the PS, and he is preparing right now.</pre>
<!-- Author: Rong-Shyang -->
<!-- Comment: Sat May 22 09;13;50 comment by...Rong-Shyang -->
<pre>Rob and I replace the PS for the b0svx00 crate. After we turned it on, I ran coldstart via RC. I also ran svxdaq with e000 and e001 and got reasonable data back. Also the SRC and VRB status pages show all GREEN light without error.</pre>
<!-- Date: Sat May 22 08:29:54 2004 -->
Tevatron lost most of the Pbar beam when switching back to the stacking lattice, see <a href=http://www-bd.fnal.gov/cgi-mcr/elog.pl?nb=2004&action=view&page=387&anchor=080105&hilite=08:01:05- target=_top> MCR elog </a>. 
Also there is a problem on MI, so we don't expect shot setup within a few hours. I had request cosmics data taking with Silicon before the b0svx00 problem happened.
<!-- Author: Rong-Shyang -->
<!-- Comment: Sat May 22 09;20;12 comment by...Rong-Shyang -->
<pre>After we fixed the PS, the quiet time is finished, so no cosmics data.</pre>



<!-- Date: Sat May 22 09:36:32 2004 -->
IFix warning for L00 East side pressure at COT face since last night, see<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=351&frame=2&anchor=08:15:06.567'> 
here </a> and <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=351&frame=2&anchor=09:16:17.194'> 
here</a>. 
There was a similar case when Steve and Rainer tried to turn on everything from zero on  <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=156&frame=2&anchor=21:46:18'> 
May 2003</a>. 
 
 
  
<!-- Author: Rong-Shyang -->



<!-- Date: Sat May 22 13:40:10 2004 -->
<center><a href='/elog/notebooks/silicon/Sat_May_22_2004_133920.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4131' ></a><a href='/elog/notebooks/silicon/Sat_May_22_2004_133927.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4133' ></a><a href='/elog/notebooks/silicon/Sat_May_22_2004_133935.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4135' ></a><a href='/elog/notebooks/silicon/Sat_May_22_2004_133941.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4137' ></a></center>
<!-- Author: Rainer -->
<!-- Comment: Sat May 22 13;42;26 comment by...Rainer -->
<b> Cosmics Running after big power outage </b>
<li> got quiet time and ran  cosmics with silicon. DAQ is happy
<li> reset the # of kicks to 8, and still no problems. Be aware next time we want to run coldstart configuration to #10 writing 0xa top 0x48000020 in b0svx02.
<li> will run a quiet time, no magnetic field dmod calibration next. 

<!-- Comment: Sat May 22 14;29;32 comment by...rainer -->
<li> ok, retreat. we have resonance errors and L1done errors during
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=351&frame=2&anchor=14:08:32.594'>
dmode </a>. Set the #of kicks back to 10. ace reports problems every 8 events  - smells like VRB buffers as pointed out by SCN earlier in the day.
<li> The # of kicks were set  from 10 to 8 May 1st. I believe we have taken dmod calibration successfully at least once during the bias scan. Rong also says he did run the cold start configuration. both must have happened witth 8 kicks enabled, apparently with no problem.
<li> let's how it goes with 10 kicks. We will have to wait until the solenoid is in a defined state to make sense out of the calibration.


<!-- Comment: Sat May 22 19;42;07 comment by...rainer -->
<li> we are running fine now. Moved RDKICKTHRES to 10 to reflect the current setting which we programmed in directly into the SRC earlier.
<!-- Date: Sat May 22 15:46:30 2004 -->
<pre><b> dmod calibration successful </b>  
with 10 kicks enabled. since b0sili01 is still down with disk problems, we need to submit the runs once the machines are back up:  
<pre> 
DPSON run 183038 
DPSOFF run 183041 
-- No beam in the machine. 
</pre></pre>
<!-- Author: rainer -->
<!-- Comment: Sun May 23 14;27;31 comment by...rainer -->
has been submitted.
<!-- Date: Sat May 22 18:38:45 2004 -->
at the beginning of store 3528, I saw several wedges in SVX go grey in the PSGui - in svxinter, correspondingly see  
<pre> 
->  Crate_info error 
 Caenet_read: Error number 7fff received 
 Crate number = 5 
 Error:  response = 32767 
 get_cr_info: ERROR, cr_n = 5 
</pre> 
it recovered without trip on its own.
<!-- Author: rainer -->


<!-- Date: Sat May 22 19:34:25 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4138&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4139  width='300' align='top'></a></td><td>SVT spymon plot for run183053, store 3528</td></tr></table>
<!-- Author: Rong-Shyang -->



<!-- Date: Sat May 22 19:59:28 2004 -->
b0sili01 had some difficulities coming back. Partition /data02 complained about inodes belonging to an orphaned list. A manual fsck fixed the problem, and b0sili01 came back in its full glory. 
However I would still advise everyone with data on /data02 to check, that everything is there.  
 
And we certainly need the UPS thingies for this machine 
  
<!-- Author: Marcel -->


<!-- Date: Sun May 23 03:48:06 2004 -->
<h2> b0fib03 blues </h2> 
<li> at 00:20am, LB1W0/W1 trip off on DVDD OverCurrent. 
<li> at the same time, run control reports 
<pre> 
(MLE) b0tsi02:Messenger:12:20:03 AM->DONE timeout 
(MLE) b0dap73.fnal.gov:Thread-26:12:20:02 AM->Done Timeout: VRB_ISL_06  
(MLE) b0dap73.fnal.gov:Thread-26:12:20:02 AM->Requested Halt-Recover-Run issued [errmon] 
(MLE) b0svx05:Messenger:12:20:04 AM->Silicon Resonator: Sl 20 Ch 3 -> e5a3: Bytes: 00086 
 
(MLE) b0svx00:Messenger:12:20:06 AM->Silicon Resonator: Sl 16 Ch 9 -> e074: Bytes: 00174 
 
(MLE) b0svx01:Messenger:12:20:06 AM->Silicon Resonator: Sl 18 Ch 2 -> e182: Bytes: 00058 
 
(RC)   0:20:05 Halt -> HALTED 
CT: 2004.05.23 00:20:10 
20'10" 1 crate/s: b0svx06(16),  Done TO.[RXPT] 
CT: 2004.05.23 00:23:06 
23'6" 1 crate/s: b0svx06(16),  Done TO.[RXPT]b0dap73.fnal.gov:Thread-26:12:23:06 AM->Failed Shepherding client FIB_ISL_03 (b0fib03) -->   
 Additional Information:  
 
 Attention!!!. FERML_SHEPHERD_FAILED !!!.  
 - Please take the necessary actions. 
 - NO MORE Auto-SHEPHERD WILL BE ISSUED  
 - until after 3 minutes of smooth running. 
[MLE] 
</pre> 
the ace shepherded the crate because it became YELLOW 
which does not help, and she aborts the run - which gets stuck as well. she finally resets. 
<li>  while in start, monitoring ace tries to reset the trips and gets persistent 
<pre> 
ERROR: Power Supply L00 WEST W1 Handshake - No response from the FIB 
</pre> 
Monitoring Ace ramps the silicon to STANDBY and at the same time LB0W4/5 trip off as well. 
<li> I saw the problems from home - spying on b0fib03, I see that it does not seem to be able to initialize FIB45/slot 16: 
<pre> 
Opening Fib slot 8 
Opening Fib slot 9 
Opening Fib slot 11 
Opening Fib slot 12 
Couldn't find Chip 5 Chip ID 132 in hdi 17 
Dumping Hdi RO 1 Addr: 17 nChips 10 
IDs:  89 88 87 86 85 84 a3 a2 a1 a0 
Thresholds:  008 008 008 008 008 000 008 008 008 008 
BitStream Length (bits) 1970 Contents: 
0000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0016: 16 2d 36 d0 06 28 8c c1 0d 00 00 00 00 00 00 00 
0032: 00 00 00 00 00 00 00 00 c0 a2 c5 16 9a 00 85 31 
0048: b8 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0064: 00 58 b4 d8 44 13 a0 30 06 37 00 00 00 00 00 00 
0080: 00 00 00 00 00 00 00 00 00 00 8b 16 db 68 02 14 
0096: c6 e0 06 00 00 00 00 00 00 00 00 00 00 00 00 00 
0112: 00 00 60 d1 62 23 4d 80 c2 18 dc 00 00 00 00 00 
0128: 00 00 00 00 00 00 00 00 00 00 00 2a 5a ec 85 09 
0144: 50 18 83 1b 00 00 00 00 00 00 00 00 00 00 00 00 
0160: 00 00 00 40 45 8b dd 30 01 0a 63 70 03 00 00 00 
0176: 00 00 00 00 00 00 00 00 00 00 00 00 a8 68 b1 1f 
0192: 26 40 61 0c 6e 00 00 00 00 00 00 00 00 00 00 00 
0208: 00 00 00 00 00 15 2d 76 c4 04 28 8c c1 0d 00 00 
0224: 00 00 00 00 00 00 00 00 00 00 00 00 00 a0 a2 c5 
0240: 9e 98 00 85 31 b8 01 
Couldn't find Chip 0 Chip ID 143 in hdi 14 
Couldn't find Chip 1 Chip ID 142 in hdi 14 
Couldn't find Chip 2 Chip ID 141 in hdi 14 
Couldn't find Chip 3 Chip ID 140 in hdi 14 
Couldn't find Chip 4 Chip ID 155 in hdi 14 
Couldn't find Chip 5 Chip ID 154 in hdi 14 
Couldn't find Chip 6 Chip ID 153 in hdi 14 
Couldn't find Chip 7 Chip ID 152 in hdi 14 
Couldn't find Chip 8 Chip ID 183 in hdi 14 
Couldn't find Chip 9 Chip ID 182 in hdi 14 
Couldn't find Chip 10 Chip ID 181 in hdi 14 
Couldn't find Chip 11 Chip ID 180 in hdi 14 
Couldn't find Chip 12 Chip ID 163 in hdi 14 
Couldn't find Chip 13 Chip ID 162 in hdi 14 
Couldn't find Chip 14 Chip ID 161 in hdi 14 
Couldn't find Chip 15 Chip ID 160 in hdi 14 
Dumping Hdi RO 1 Addr: 14 nChips 16 
IDs:  8f 8e 8d 8c 9b 9a 99 98 b7 b6 b5 b4 a3 a2 a1 a0 
Thresholds:  000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 
BitStream Length (bits) 3152 Contents: 
0000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0032: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0048: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0096: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0112: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0128: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0144: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0176: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0192: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0208: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0224: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0256: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0272: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0288: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0304: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0336: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0352: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0368: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
0384: 00 00 00 00 00 00 00 00 00 00 
<b> 
Opening Fib slot 16 
FISION: S_errno_EIO (0x5): VME BERR received! 
FISION: S_errno_EIO (0x5): VME BERR received! 
FISION: S_errno_EIO (0x5): VME BERR received! 
checkSum: VISION read error = 0x5 at 0x240000 
Bad Seq Ram Slot 16 IDO f809 Calc 000003fc Expect 00000000 
FISION: S_errno_EIO (0x5): VME BERR received! 
[...] 
checkSum: VISION read error = 0x5 at 0x240000 
Bad EEPROM  Slot 16 IDO f809 Calc 000003fc Expect 00000000 
FISION: S_errno_EIO (0x5): VME BERR received! 
[...] 
FISION: S_errno_EIO (0x5): VME BERR received! 
 
Failed to Initialize Fib: Error 0x00000005 
Mess Error Type 5 Num 273 
</b> 
Opening Fib slot 17 
 
Opening Ffo slot 14 
Error in Msg Initialization: 00000005 
 
Exiting Loop because end of message 
Not Using Broker Right Now 
Publishing Init Config... 
FISION: S_errno_EIO (0x5): VME BERR received! 
FISION: S_errno_EIO (0x5): VME BERR received! 
</pre> 
<li> in addition, in SVXDAQ for this slot16 only  the CommandFifoError LED is RED (see below) 
<li> mapvme reports 
<pre> 
 
b0fib03 VmeBusScan 
  PartitionId:        0 
  IP: 131.225.237.123  Ethernet: 08:00:3e:28:c7:3e 
  SystemNumber:       S07953|3410489|549635 
  CpuType:            Motorola MVME2300 - MPC 603p 
  Slot:  8  Id:   108   2 FIB Fiber Interface 
  Slot:  9  Id:   123   2 FIB Fiber Interface 
  Slot: 11  Id:   165   2 FIB Fiber Interface 
  Slot: 12  Id:   107   2 FIB Fiber Interface 
  Slot: 14  Id:  550305   4 FFO FIB Fanout 
  Slot: 17  Id:   150   2 FIB Fiber Interface 
</pre> i.e. does not see Slot 16 
<li> I decide to take out the two wedges LB0W0/W1 corresponding to this FIB, as L00 is compromised anyway with the blown crowbar fuses. 
<li> unfortunately, I cannot the the handshake to work even after several reboots. 
<li> so I instruct the aces to take out b0fib03 and take data until I come in. 
<li> Even with the putative offender FIB masked off, the bus error messages persist and no handshake is possible. Of course with out handshake, switching on the ladders is impossible and therefore the coldstart of the crate fails. 
<li> it occured to me that maybe the hardware database info gets read only on PARTITITON instead of CONFIG, I ask DAQ/Ray Culbertsion and he confirms my suspicion. 
<li> take b0fib03 back in data taking and partition, but handshake would still fail. 
<li> I decide to take out the other two wedges LB0W4/5 out - with this no other wedge needs powering and finally I get b0fib03 to pass transitions.  
<li> we are running now with LB0W0/1 and LB0W4/5 off but otherwise ISL and SVX are fine. 
<li> there are still bursts of  
<pre> 
FISION: S_errno_EIO (0x5): VME BERR received!                                
</pre>    
I still hear SCN's voice in the back of my head that this does not mean anything but I don't know why - maybe because L00 in readall produces more data traffic     and therefore more likelyhood for bus errors.  anyway it remains odd that the handshake would not work even after slot 16 was out. 
<li> FIB45 and FIB46 are now removed from the hwdb.                   
<!-- Author: Rainer -->


<!-- Date: Sun May 23 03:49:29 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4140&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4140  width='300' align='top'></a></td><td>SVXDAQ screenshot with slot 16/FIB45 bananas.</td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Mon May 24 08;02;50 comment by...SCN -->
Every bit that can be set is set to "1" (look at the module ID, etc) which indicates on every VME read to the FIB you get back "ffffffff".  It is not surprising that nothing works.  I suggest
this is a blown fuse on the FIB, which you can confirm with an access and looking at the lights on the front panel before turning
off the crate.<P>
PS  BERR does mean something.  Not sure what my voice in Rainer's head meant.
<!-- Comment: Mon May 24 08;37;33 comment by...rainer -->
<b> Voice found </b>
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=168&frame=2&anchor=18:38:59'>
here </a> but I guess unrelated to this case. We get still FISION BERR on b0fib03 even with the offending FIB masked - we asked for an access but got turned down by ops due to the crowbar fuse situation.
<!-- Date: Sun May 23 14:27:20 2004 -->
<p><font size=+1> D-mode Calibration Job Started </font>
<p><pre>
Job id            : 341
Calibration type  : nonstandard
Run period tag    : May_2004
DPS-off run       : 183038
DPS-on run        : 183041
Requested by      : Rainer Wallny
Main process id   : 75897882
Comment           : No Beam in machine
</pre>
<!-- Author: CDF silicon D-mode calibration job server v1.4 -->



<!-- Date: Sun May 23 14:28:10 2004 -->
<p><font size=+1> D-mode Calibration Job Finished </font>
<p><pre>
This is a computer-generated silicon D-mode calibration status report.
Silicon D-mode calibration job 341 FAILED.

Job info
--------
Calibration type : nonstandard
DPS-off run      : 183038
DPS-on run       : 183041
Requested by     : Rainer Wallny
Submitted        : 2004-05-23 14:27:12
Completed        : 2004-05-23 14:28:06
Status           : FAILURE
Comment          : No Beam in machine

Last processing stage reported : checking silicon configuration
Failure diagnostics :
Inconsistent dps mode for run 183038. Check your argument order.
Exiting.
</pre>
<!-- Author: D-mode script run_calib_analysis.tcl -->


<!-- Date: Sun May 23 14:39:25 2004 -->
<p><font size=+1> D-mode Calibration Job Started </font>
<p><pre>
Job id            : 342
Calibration type  : standard
Run period tag    : May_2004
DPS-off run       : 183041
DPS-on run        : 183038
Requested by      : Jason Nielsen
Main process id   : 75939608
</pre>
<!-- Author: CDF silicon D-mode calibration job server v1.4 -->



<!-- Date: Sun May 23 19:27:02 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4143&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4144  width='300' align='top'></a></td><td>The whole SVX triped due to SRC error.</td></tr></table>
<!-- Author: Rong-Shyang -->
<!-- Comment: Sun May 23 19;36;00 comment by...Rong-Shyang -->
I was on the phone with ACE and tried to bring back the whole SVX and through coldstart. At the same time, Tevatron als wanted to dump the beam immediately, so everything is like losing control.
Also unfortuntely we lost the svxinter log due to the standby command was issued. See also<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=355&frame=2&anchor=19:04:51.408'>
elog</a>.
Several ladders kept tripping for a while but eventually, everything came back and pass the config. The first time we want to activate the run, we got 
<pre>
---
(RC)   18:14:11 Activate -> ACTIVE
(MLE) b0dap42.fnal.gov:Thread-26:6:14:14 PM->FrontEnd Error: VRB_SVX_02 
CT: 2004.05.23 18:14:15
14'15" 1 crate/s: b0svx02(112),  in error.[RXPT]
(TS)   18:14:51 ********* L2 trigger rate has dropped to 0 Hz! Events are not being logged !
(MLE) b0svx03:Messenger:6:15:02 PM->Silicon Resonator: Sl 10 Ch 9 -> e334: Bytes: 00050

(MLE) b0svx04:Messenger:6:15:02 PM->Silicon Resonator: Sl 10 Ch 4 -> e424: Bytes: 00102

CT: 2004.05.23 18:15:03
15'3" 1 crate/s: b0svx02(112),  in error.[RXPT]b0svx02:Messenger:6:15:03 PM->SRC Fatal Error:Sl 5 L2A w/o L1A 
 -->  
 Additional Information: 

 Attention !!!. FERML_SRC_FATALITY ERROR !!! 
 SRC Fatal Error from b0svx02: Sl 5 L2A w/o L1A 
 
 - HALT the run                   
 - record any relevant information in e-log     
 - RECOVER - RUN     
[MLE]
(RC)   18:15:04 End -> IDLE
---------- </pre>

so we couldn't activate. After I did 'init system' from svxdaq, I can activate the coldstart running again.






<!-- Comment: Sun May 23 20;20;47 comment by...Sal, Marcel, Rong-Shyang -->
<pre>After consulting with Sal, we think that there is something wrong with SRC clock. Unfortunately, we don't have scope trace.



</pre>
<!-- Comment: Sun May 23 21;03;05 comment by...Marcel -->
Similar incidents are <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=171&frame=2&anchor=09:55:39'>
here </a>
and
<a href=
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=190&frame=2&anchor=09:50:01'>
here </a>
The famous SRC Schlachtfest is in the part with reconstructed elogs.
<!-- Date: Sun May 23 19:37:32 2004 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4145&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4145' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4146&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4147' ></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4148&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4149' ></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4150&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4151' ></a></center>
<!-- Author: Rong-Shyang -->
<!-- Comment: Sun May 23 19;38;56 comment by...Rong-Shyang -->
<pre>DVDD history plots for some SVX ladders. The DVDD current dropped suddently instead of tripped on overcurrent.</pre>



<!-- Comment: Sun May 23 21;05;11 comment by...rainer -->
<pre>we only have readback every 7 secs.  a SPT you might be able to catch since it allows overcurrent for about 7 readings or so, but a hard trip is too fast - you catch them by luck.</pre>

<!-- Date: Sun May 23 21:15:21 2004 -->
<p><font size=+1> D-mode Calibration Job Finished </font>
<p><pre>
This is a computer-generated silicon D-mode calibration status report.
Silicon D-mode calibration job 342 FAILED.

Job info
--------
Calibration type : standard
DPS-off run      : 183041
DPS-on run       : 183038
Requested by     : Jason Nielsen
Submitted        : 2004-05-23 14:38:58
Completed        : 2004-05-23 21:13:53
Status           : FAILURE

Last processing stage reported : starting histogram transfer
Failure diagnostics :
You have no controlling tty and no DISPLAY.  Cannot read passphrase.
lost connection
</pre>
<!-- Author: D-mode script run_calib_analysis.tcl -->



<!-- Date: Sun May 23 21:29:52 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4153&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4153  width='300' align='top'></a></td><td>The probability of getting a L2 without L1 as a fraction of the run time. 
The logfile starts from 17.04.2004. 
It seems, we get most of the errors at the start  of a run  
 
</td></tr></table>
<!-- Author: Marcel -->


<!-- Date: Sun May 23 22:23:33 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4154&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-4155  width='300' align='top'></a></td><td>SPT spymon plot for store 3530, run 183077</td></tr></table>
<!-- Author: Rong-Shyang, Marcel -->


<!-- Date: Sun May 23 22:32:41 2004 -->
there was a busy time out in  run  
<a   href='http://www-cdfonline.fnal.gov/java/cdfdb/servlet/ShowFile?FILE=/runSummary/errorlog/errorfile183077.log'> 183077 </a> 
and it took b0svx00 quite a while to recover (stayed yellow). I vxlogined into it and still caught 
<pre> 
FISION: S_errno_EIO (0x5): VME BERR received! 
Sent ack to Run Control: 0 HALTED SUCCESS 
Error: Vrb Timeout BUSY- Slots:  12:e040 
 
Error: Vrb Timeout !DONE- Slots:  12:e040 
Password: Error: Vrb Resonance Candidate Sl 16 Ch 9 -> e074: Bytes: 00186 
 
Messenger: processStatechange Recover [0] 
Sent ack to Run Control: 0 RECOVERED SUCCESS 
 
b0svx00-> Messenger: processStatechange Run [0] 
Sent ack to Run Control: 0 ACTIVE SUCCESS 
</pre> 
The luminosity is quite high - are we seeing first trouble with readout time ?
<!-- Author: rainer -->
<!-- Comment: Mon May 24 08;08;57 comment by...SCN -->
Bet you buss erred on Slot 12, which is why it reports
both BUSY and !DONE, and why it took "a long time".  I don't
believe it has anything to do with luminosity.  <ol>
<li>Is this the same crate where the BUSY timeout was generated?
<li> Did it recover without intervention or did the EVB need cleaning up or something
<li> Is there a reason you suspect readout time, or is just a random guess?  Looks like the biggest byte count is 186 from this crate, which is far far far far far below the 8k buffer size limit, and only 3.7 uS readout time-not at all excessive.
</ol>
My suspicion is that either the VRB VME interface hung up or the EVB had some problem with the VME bus in this crate.


<!-- Comment: Mon May 24 08;39;46 comment by...rainer -->
<pre>&lt;pre&gt;
1. yes 
2. no, recovered by itself
3. random guess.
thx.
&lt;/pre&gt;</pre>

<!-- Date: Mon May 24 10:17:31 2004 -->
<p><font size=+1> D-mode Calibration Job Finished </font>
<p><pre>
This is a computer-generated silicon D-mode calibration status report.
Silicon D-mode calibration job 342 FAILED.

Job info
--------
Calibration type : standard
DPS-off run      : 183041
DPS-on run       : 183038
Requested by     : Jason Nielsen
Submitted        : 2004-05-23 14:38:58
Completed        : 2004-05-24 10:17:00
Status           : FAILURE

Last processing stage reported : comparing to the reference calibration
Failure diagnostics :
The calibration failed the following reference comparison criteria:

            Cut Name    Sensor Set   Cut Value   Data Value
--------------------    ----------   ---------   ----------
bad_fraction abs max    SVX_PHI      0.03        0.030401  

Reference calibration runs are 174253 (DPS off) and 174252 (DPS on)
</pre>
<!-- Author: D-mode script run_calib_analysis_noph.tcl -->