<!-- Title: 08:55:26  Mon Jun  9 2003 -->
<!-- Crewchief: Rainer -->
<!-- Op1: everybody else -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
New week, new file. hopefully no more fuses blown.

<!-- Date: Mon Jun 9 12:22:39 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: IB1W4L1 
<li>problem n: 576 
<li>Status: needs_observation
<li>Type: DropReadOut_nonRX
<li>Action: Monitoring
<li>entered by: tompkins 
<li>description is: HDI has no errors for data runs but 100% errors for calibration runs due to c2c2 terminations.   What happens differently in calibrations runs that would cause these errors which don't show up in data? 
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B1W4&layer=L1">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Mon Jun 9 13:08:20 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Mon_Jun__9_2003_130753.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2139  align='top'></td><td>le SVT plot du jour. pas de troux inconnu. </a></td></tr></table>
<!-- Author: rainer -->


<!-- Date: Mon Jun 9 18:27:32 2003 -->
why does the bit reporter show f142 as dark grey since run 164353 ? See  <a href='http://nkek15.fnal.gov/cgi-bin/errmon/makeladder.pl?beginrun=162705&endrun=164381&ladder=I142'> here </a>. It's in the hardwarede database.
<!-- Author: rainer  -->


<!-- Date: Mon Jun 9 20:51:34 2003 -->
<p> 
Dear SVX'ers: 
<p> 
Please see 
<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=svt&action=view&page=7&frame=2&anchor=20:33:39'> 
SVT e-log</a>   
for my detective work trying to track down some SVT/svtsim 
mismatches in <a href=" 
http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W1&layer=L3 
">SB2W1L3</a>.  I suspect that a VTM swap may be in order 
(though in principle it could be VRB, VTM, or VRB backplane, 
but the VRB was already swapped a while back, and a backplane 
problem is too icky to want to consider.) 
<p> 
Your quasi group member, 
<p> 
  
<!-- Author: Bill Ashmanskas -->
<!-- Comment: Wed Jun 11 08;59;32 comment by...SCN -->
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=08:55:08'>Swapped VTM</a>
<!-- Date: Tue Jun 10 02:02:17 2003 -->
<pre>At 00:38 iFIX reported a trip in the SVX. Since the accelerator people were going to end the store we weren't taking data; moreover, we were in Start state in RC. Anyway, the error was DVDD OverCurrent in SVX B3, layers 1, 2 and 3. I did Reset Trips and at 01:12 the Silicon GUI wrote 'success: Resetting trips'. 
 
But the color of those three layers was still red, and their HV was off. I turned off the HV of the whole SVX detector, something that made another trip and alarm from iFIX. After Reset Trip and setting the whole HV Stand By all the layers are now in yellow or blue state in the Silicon GUI. 
 
In IMON almost all the SVX AVDD chips are pink due to OverCurrent. The values are around 10 mA, small compared to the currents, that are ~500 mA. We don't plan to use the Silicon detectors during the rest of the shift.</pre>
<!-- Author: Jonatan -->
<!-- Comment: Tue Jun 10 07;58;32 comment by...rainer -->
I should add that I had the ace on the phone and told him to proceed like that. the overcurrent in AVDD is due to the fact
that after a (accidental) power cycle of SVX all chips come up
in power up state which draws slightly more current. this is taken care of by a  cold start but silicon at the time was out of readout. I'll do this stand alone momentarily.
NB: The AVDD currents were reported to be  some few 10mA _above_ the nominal threshold.

<!-- Comment: Tue Jun 10 08;52;21 comment by...rainer -->
BTW: the trip was B3W8 L1-3.


<!-- Date: Tue Jun 10 11:27:58 2003 -->
got another crate communication error like the one earlier  
<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=157&frame=2&anchor=01:28:29'> 
this </a> week. IFIX reported communication loss with CAEN crate #7.  The monitoring ace and steve performed what we should call  
the 'hocker manoeuver' until andy comes up with a better name. anyway, they 'hockered' the crate,switched L00 and ISL to standby in ifix. L00 B0 W0/5 came back in trip state, so they had to manually ramp the two wedges to standby. Steve observed a spike 
in the background roughly at the same time the trip happened. beam related hiccoughs ?
<!-- Author: rainer,for steve et al. -->
<!-- Comment: Tue Jun 10 12;30;43 comment by...Andy H. -->
Time to update
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=153&frame=2&anchor=14:46:05'>
this</a> table:
<p>
<center><table BORDER COLS=3
WIDTH="500" NOSAVE > <tr> <td>Date</td> <td>Failure mode</td> <td>Crate number
(1-16)</td> </tr> <tr> <td>12-MAR-2003</td> <td>Communication failure</td> <td>16</td> </tr>
<tr> <td>27-MAR-2003</td> <td>Spontaneous reboot</td> <td>2</td> </tr> <tr>
<td>02-APR-2003</td> <td>Communication failure</td> <td>15</td> </tr> <tr>
<td>13-APR-2003</td> <td>Spontaneous reboot</td> <td>6</td> </tr> <tr> <td>16-APR-2003</td>
<td>Communication failure</td> <td>1</td> </tr> <tr> <td>03-MAY-2003</td> <td>Spontaneous
reboot</td> <td>13</td> </tr> <tr> <td>07-MAY-2003</td> <td>Spontaneous reboot</td> <td>4</td>
</tr> <tr> <td>09-MAY-2003</td> <td>Spontaneous reboot</td> <td>8</td> </tr>
<tr><td>08-JUN-2003</td> <td>Communication failure</td> <td>12</td> </tr>
<tr><td>10-JUN-2003</td> <td>Communication failure</td> <td>7</td> </tr>

 </table></center> 
</p>


<!-- Date: Tue Jun 10 12:31:01 2003 -->
test
<!-- Author: Andy H. -->
<!-- Comment: Tue Jun 10 12;41;02 comment by...Rainer -->
test succeeded. thanks for update of the table. that give's a feel of what's going on.


<!-- Date: Tue Jun 10 13:54:38 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Tue_Jun_10_2003_135434.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2141  align='top'></td><td>Still seeing that the BXING number is not getting into the Data with the 2nd SRC operating in Tracer Mode.  Here is what it looks like on the backplane of the Fib crate.  Traces to look at are BXING, FCMD (Fib Commands, you can see DIGITIZE and READOUT) and SYNCH, which is the 132 ns Clock.</a></td></tr></table>
<!-- Author: SCN -->


<!-- Date: Tue Jun 10 16:29:52 2003 -->
<p> 
Another CAEN failure <b>in the same day</b> -- udapte AGAIN: 
</p> 
<center><table BORDER COLS=3 WIDTH="500" NOSAVE > <tr> <td>Date</td> 
<td>Failure mode</td> <td>Crate number (1-16)</td> </tr> <tr> <td>12-MAR-2003</td> 
<td>Communication failure</td> <td>16</td> </tr> <tr> <td>27-MAR-2003</td> <td>Spontaneous 
reboot</td> <td>2</td> </tr> <tr> <td>02-APR-2003</td> <td>Communication failure</td> 
<td>15</td> </tr> <tr> <td>13-APR-2003</td> <td>Spontaneous reboot</td> <td>6</td> </tr> <tr> 
<td>16-APR-2003</td> <td>Communication failure</td> <td>1</td> </tr> <tr> 
<td>03-MAY-2003</td> <td>Spontaneous reboot</td> <td>13</td> </tr> <tr> 
<td>07-MAY-2003</td> <td>Spontaneous reboot</td> <td>4</td> </tr> <tr> <td>09-MAY-2003</td> 
<td>Spontaneous reboot</td> <td>8</td> </tr> <tr><td>08-JUN-2003</td> <td>Communication 
failure</td> <td>12</td> </tr> <tr><td>10-JUN-2003</td>  
<td>Communication failure</td> <td>7</td> </tr> 
<tr><td>10-JUN-2003</td> <td>Communication failure</td> <td>16</td> </tr> 
 </table></center> 
</p> 
<p>What the hell is going on?</p>
<!-- Author: Andy H. -->
<!-- Comment: Tue Jun 10 17;10;56 comment by...David, Steve, Rainer -->
Hockering this guy (crate #16) did not work this time. we tried several times, with an SPL, a rookie pager carrier and an expert pager carrier shadow present. As a last resort, we powered off all the ladders corresponding to caen crate #15 and #16 (L00 EAST W0,4,5 IB5W4,IB3W2,IB5W4) and power cycled
the rack SVNEB from the cryo area which took 10min. Communication could be restablished, we powered back to standby using IFIX and checked that the corresponding fib b0fib07 did resync. IB5W4 tripped on DVDD but we got it back from the gui. We got back just in time when CDF started data taking. 



<!-- Comment: Tue Jun 10 17;30;03 comment by...rainer -->
Shortly after, gui freezes and needs rebooting (at about 15:45).
<!-- Comment: Tue Jun 10 19;12;49 comment by...rainer -->
Hockerization procedure described <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=157&frame=2&anchor=01:28:29'>
here </a>.
<!-- Date: Tue Jun 10 17:23:34 2003 -->
SVXMon showed SB1W10L4 yellow in this store. We don't anything 
in SVTMon plot but this is now 4 out of 5 so that does not mean anything. We see <p> 
e1a4 4dfc 00f3 00f3 00f3 00f3 00c2 c2c4 e1b0 4dfc 8317 00f3 
     2205 230a 24f3 250b 2606 8217 a117 a017 
<p> 
in the data. so f3's. In the meantime, SVXMon turns SB1W0L4 green (?) but the latest data still shows f3's. Currently SVX error 
logger unavailable but will check later.
<!-- Author: Rainer, David -->


<!-- Date: Tue Jun 10 17:24:06 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Tue_Jun_10_2003_172357.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2143  align='top'></td><td>svt du jour.</a></td></tr></table>
<!-- Author: rainer -->



<!-- Date: Tue Jun 10 17:43:01 2003 -->
While I am in the control room, TEVMon produces two warnings within  30mins <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=483&frame=2&anchor=17:15:13.469'> 
here </a> and <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=483&frame=2&anchor=17:40:11.815'> 
here </a>. In the first case there was a continous increase in lostp, in the scond case a spike rms(lostp). 
<!-- Author: rainer -->


<!-- Date: Tue Jun 10 17:46:00 2003 -->
Kaori works on a new consumer version. there is a new SVXMon  
version and a new Display version. In the new-new configuration, 
there are occasional all red areas, i.e. whole of L00 or SVX turns red and comes back.  I look at SVT plot and error logger 
and think this is not real. It looks like there is a problem  
with the different new versions. it appears the old monitor version works fine with the new display version, since both new and old display version appear coherent. However, in the old mon -> new display configuration the spontaneous 'common mode' reds re-appear.
<!-- Author: rainer -->


<!-- Date: Tue Jun 10 17:55:54 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Tue_Jun_10_2003_175544.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2147  align='top'></td><td>this is the 'common mode' red I was talking about.</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Tue Jun 10 18;08;41 comment by...rainer -->
Kaori goes back to the old-old version. If the display fails,
by default the new display will show up, but the current CO
knows how to get the old display. No reds seen so far.



<!-- Date: Tue Jun 10 18:20:27 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Tue_Jun_10_2003_182023.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2151  align='top'></td><td>this is the corresponding stripmap to the HRR reported in  general elog  </a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Tue Jun 10 18;29;34 comment by...rainer -->
2 HRRs from in a row  from SB0W3L4 <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=483&frame=2&anchor=18:05:36.405'>
here </a>. According to PDB its a known jumper failure and steve suggested already masking some chips. Which we should probaby
do since it already acted up 
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=157&frame=2&anchor=10:33:43'>
earlier </a>. 



<!-- Date: Tue Jun 10 18:44:42 2003 -->
<center><a href='/elog/notebooks/silicon/Tue_Jun_10_2003_184333.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2153' ></a><a href='/elog/notebooks/silicon/Tue_Jun_10_2003_184404.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2155' ></a></center>
<!-- Author: Rainer -->
<!-- Comment: Tue Jun 10 18;55;24 comment by...rainer -->
With SVXMon back in business, I can finally check 
SB1W10L4 occupancies. Undoubtedly, there is data coming from 
this ladder. Looking at hex dump, I confirm this. strange.




<!-- Comment: Thu Jun 12 07;44;38 comment by...rainer -->
Maybe a DSP OFF mad cow incident?
<!-- Date: Wed Jun 11 07:40:23 2003 -->
<pre>&lt;h3&gt;Silicon OFF for Aperture Scans&lt;/h3&gt; 
See &lt;a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=484&frame=2&anchor=06:18:27.572'&gt; 
main elog&lt;/a&gt; for discussion. </pre>
<!-- Author: SCN -->
<!-- Comment: Wed Jun 11 07;41;31 comment by...one who knows to click the HTML button -->
<h3>Silicon OFF for Aperture Scans</h3> 
See <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=484&frame=2&anchor=06:18:27.572'> 
main elog</a> for discussion.

<!-- Comment: Wed Jun 11 09;53;29 comment by...SCN -->
PS - I disabled the global interlock Kill Switches to be sure there is no illicit powering, spontaneous or otherwise. 
<!-- Date: Wed Jun 11 08:55:08 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W0/1All 
<li>problem n: 451 
<li>Status: needs_observation
<li>Type: Other
<li>Action: Swap_VRB
<li>entered by: nahn 
<li>description is: Swap VTM for these Wedges due to <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=20:51:34'>SVT request</a> regarding  <a href='http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W1&layer=L3'>SB2W1L3</a><P> VTM In:  PREP 560748<BR> VTM Out: PREP 561034<BR> No bent pins seen on rear side of J3 backplane.<table border=1>
<tr><td><b>IN</b></td><td>PREP:</td><td>560748 (VTM)</td><td>Serial:</td><td>??</td></tr>
<tr><td><b>OUT</b></td><td>PREP:</td><td>561024 (VTM)</td><td>Serial:</td><td>??</td></tr></table> 
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W0/1&layer=All">report</a></ul>
<!-- Author: Problem DB -->




<!-- Date: Wed Jun 11 10:58:58 2003 -->
<p><font size=+1> D-mode Calibration Job Started </font>
<p><pre>
Job id            : 264
Calibration type  : standard
Run period tag    : Jun_2003
DPS-off run       : 164105
DPS-on run        : 164104
Requested by      : Jason Nielsen
Main process id   : 134249520
</pre>
<!-- Author: CDF silicon D-mode calibration job server v1.3 -->



<!-- Date: Wed Jun 11 16:12:12 2003 -->
persistent pinky on SB5W0L1. probably needs current adjustment. I'll watch it.
<!-- Author: rainer  -->

<!-- Date: Wed Jun 11 16:18:31 2003 -->
<p><font size=+1> D-mode Calibration Job Finished </font>
<p><pre>
This is a computer-generated silicon D-mode calibration status report.
Silicon D-mode calibration job 264 FAILED.

Job info
--------
Calibration type : standard
DPS-off run      : 164105
DPS-on run       : 164104
Requested by     : Jason Nielsen
Submitted        : 2003-06-11 08:57:49
Completed        : 2003-06-11 14:17:12
Status           : FAILURE

Last processing stage reported : comparing to the reference calibration
Failure diagnostics :
The calibration failed the following reference comparison criteria:

           Cut Name    Sensor Set   Cut Value   Data Value
-------------------    ----------   ---------   ----------
dps_off_ped abs max    SVX_PHI      0.5         0.957478  
 dps_on_ped abs max    SVX_PHI      0.2         0.350632  
 dps_on_ped abs max    ISL_PHI      0.25        0.301369  
 dps_on_ped abs max    ISL_Z        0.25        0.269161  
   bad_chan rel max    SVX_Z        0.25        0.25524   

Reference calibration runs are 159187 (DPS off) and 159186 (DPS on)
</pre>
<!-- Author: D-mode script run_calib_analysis.tcl -->



<!-- Date: Wed Jun 11 18:34:42 2003 -->
<center><a href='/elog/notebooks/silicon/Wed_Jun_11_2003_183227.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2157' ></a><a href='/elog/notebooks/silicon/Wed_Jun_11_2003_183421.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2159' ></a></center>
<!-- Author: Rainer -->
<!-- Comment: Wed Jun 11 19;14;56 comment by...rainer -->
DVDD currents of SB5W0L1 (see comment <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=16:12:12'>
earlier </a>). Caution Range is 89.000 <-->  121.000. left plot shows current of the last stores, right plot currents in this store (include today's data does not work for me in DS plotter). There is no entry in the PDB related to this wedge and it is otherwise behaving fine (checked occupancy in svxmon and error messages on svx error logger)
Decide to temporarily adjust this one for caution range 128 since
the it looks like the chips draw more current after they have been off for a while.  I have not found david's examples in
the PDB so I hope my values are ok. rename current lookup.txt
to lookup.txt.davids_table.

<!-- Date: Wed Jun 11 18:36:40 2003 -->
<h1>Silicon Off Day Activities</h1> 
<h3>Testing PS GUI on b0dap44</h3> 
A new computer (b0dap44) has been provided to replace b0dap42, 
which is old and slow.  However, it is running RH 7.3.1, and 
as such sends to the VME controller directories such as  
<i>/autofs/mnt/cdf/people2/svxii/daq/PSAnywhere</i> rather than 
just <i> /cdf/people2/svxii/daq/PSAnywhere </i> and this 
the VME controller cannot handle.  Sergeui attempted to 
fix the problem by making soft links on b0dap24 which is the host for the VME CPUs, but we didn't have a chance to test it. <B>This is the perfect thing to test using the Test stand GUI setup</b> so that is the next step 
<h3>Testing the Hockering of crates</h3> We attempted to 
reset CAEN crates 15 and 16 by pressing the reset button, but 
there was no loss of communication between the crates and the GUI, so presumably neither crate was in fact reset.  Next investigation is in the pit. 
<h3>2 SRC tests</h3> 
Lester and I first ran a new 2 SRC Prom in the diagnostic crate,  
and saw that both the BXING problem is fixed and the SRC does not 
send the SCAN commands to the VRB until the <i>real L2A</i> rather than the "express" L2A, which means the SRC should get 
the Event ID correct.  We then jury rigged the system to have one 
Fib driven by the old SRC (no ladders on, just FIb header and truncate) and the teststand ladder driven by the 2nd SRC in b0svx06 with the new prom.  We took several runs with Software and Hardware Event Builder and saw no errors, and that the Bunch Crossing numbers matched, which was not true <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=154&frame=2&anchor=07:44:21'>in previous tests</a>.  Armed with confidence, we asked to start the next store with the 2nd SRC playing with the TS but not driving any 
silicon, but this failed because the TDCs and possibly SVT did not like the "express L2A" message coming from the TS.  In principle <B>Two SRC Mode looks good for Silicon</b> but the issues with the other systems will need to be understood before the next attempt 
<h3>53 MHz Clock fanouts installed</h3> Steve Chappa installed 2 more 53 MHz clock fanouts.  I told him we would test them 
against the existing one, and if they are fine we would use 
them instead of the existing ones, which are just kludges anyway 
and he needs to recover for spares.  The testing didn't get done yet, but having them there is certainly required. The Test SRC is now connected to one of these new ones. 
<h3>Turned the Silicon Back On</h3> 
Of course, as the shot came up we did turn all the Silicon back on.  It went rather smoothly except that a few ladders came back with slightly higher AVDD than before.  David C adjusted the IMON parameters.<P> 
 
We are now in a shot, and started with two SRC mode but got quickly taken out.
<!-- Author: SCN -->


<!-- Date: Wed Jun 11 19:25:35 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB5W0L1 
<li>problem n: 1060 
<li>Status: needs_observation
<li>Type: DVDD
<li>Action: IMON_limits
<li>entered by: rainer 
<li>description is: adjusted upper limit of good range from 116 to 128 to allow  for increased currents after power off.  
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B5W0&layer=L1">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Wed Jun 11 19:45:05 2003 -->
I just tried to get a candy from the candy machine and the candy machine died. what <i> <b> is </b> </i> going on ?
<!-- Author: rainer -->


<!-- Date: Wed Jun 11 19:47:47 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Wed_Jun_11_2003_194733.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2161  align='top'></td><td>SVT plot du soir .... what the f .... ???</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 20;19;33 comment by...rainer -->
SB0W4/5 was accidentially left marked off in the hardware database during SRC2 study. As a consequence, SVXMon marks corresponding
ladders as blue, so CO does not notice as once (nor does sili expert). currents acted up, but since we had to power off abnormal currents were expected so the problem got shrouded 
by that (another reason not to power off light-heartedly, I guess.) Only visible in above SVT plot. I get the corresponding
FIB_09 online, make the DAQ Ace reload the hardware database (end run -> refresh hardware database -> config -> run) 


<!-- Date: Wed Jun 11 20:24:02 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Wed_Jun_11_2003_202353.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2163  align='top'></td><td>SVT plot du jour - meilleur!</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 20;24;59 comment by...rainer -->
marking FIB_09 online and refreshing the hardware database
did the job. I ask CO to restart SVXMon.


<!-- Date: Wed Jun 11 20:43:54 2003 -->
<pre>How do I get the problem database reload/update from the hardware database ?</pre>
<!-- Author: rainer -->


<!-- Date: Wed Jun 11 21:12:11 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Wed_Jun_11_2003_211201.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2167  align='top'></td><td>SB2W9L2 is dropping readout w/ c2c2</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 21;48;01 comment by...rainer -->
chip 0 reports pipeline out of sync and rest of chips show no data also bit 0/1 errors.



<!-- Date: Wed Jun 11 21:49:04 2003 -->
power cycle SB2W9L2 to see whether we can get data back ... 
(chip 88 was not giving data 'since beginning of time' but  otherwise ok ladder)
<!-- Author: rainer -->


<!-- Date: Wed Jun 11 21:51:10 2003 -->
IBOW0L0 produces stuck cell ID ....
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 21;51;34 comment by...rainer -->
HRR I meant to say ....


<!-- Date: Wed Jun 11 21:59:28 2003 -->
<center><a href='/elog/notebooks/silicon/Wed_Jun_11_2003_215905.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2169' ></a><a href='/elog/notebooks/silicon/Wed_Jun_11_2003_215913.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2171' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 22;15;36 comment by...rainer -->
Has a history of producing bit 0/1 errors ...
<a href=
'http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B0W0&layer=L0'>
PDB entry </a> 

<!-- Comment: Wed Jun 11 22;36;45 comment by...rainer -->
above refers to HRRs in IBOW0L0 ... I conclude these are not the dangerous ones we are looking for. In fact have not seem one in some time now ...


<!-- Date: Wed Jun 11 22:40:40 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Wed_Jun_11_2003_224033.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2177  align='top'></td><td>powercycling SB2W9L2 did not help ....</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 22;52;47 comment by...rainer -->
this is what I see in recent A stream data in subsequent bunch crossings (run 164512)

e292 4dfc c940 0000 e293 4dfc 8920 00f3 
e292 7ffc c97d 0000 e293 7ffc 8903 ....

in the SVX error logger I see only one chip  C1??? (there used to be a lot of unhappy faces as well ...)

they stopped the run so I look at the new run 164517 ...


<!-- Date: Wed Jun 11 23:09:56 2003 -->
<pre>e292 47fc 8909 00f3 8809 8709 30f3 3110 3217 3339 3439 3509 3606 3809 390c 3a1e 3b2a 3c42 3d08 7808 7953 7a0c 7bf3  
8609  
8509 
a409 a309 7404 7510 7632 773b 7810 7903 
a209 2908 2a17 2b45 2c34 2d10 2e06 2f0e 3008 310e 3216 3329 342e 3526 3609 7703 
780e 792c 7a24 7b08  
a109 a009 7ff3 c1c1 0000 
-------------------------------------</pre>
<!-- Author: rainer -->


<!-- Date: Wed Jun 11 23:25:51 2003 -->
<pre>at 23:00h there is a TEVMon warning about abort gap losses. Silicon comes to standby at 23:06h. I ask the SciCo to contact 
MCR.  It appears that MCR is not aware of an incident. I want to see the B0PAGC plot before proceeding .... &lt;p&gt;  
MCR say they are not aware of anything ... but we see a huge spike in B0PAGC ... I page Steve. 
 
</pre>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 23;35;41 comment by...rainer -->
The spike is <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=486&frame=2&anchor=23:22:56.151'>
here </a> 


<!-- Comment: Wed Jun 11 23;37;51 comment by...rainer -->
We see all sorts of other  <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=486&frame=2&anchor=23:31:00.652'>
stuff </a> going on, but Steve says that
if the relevant parameters TevMon uses are ok (which they now are) we can proceed. Ramping up HV and 23:29h.


<!-- Date: Wed Jun 11 23:49:36 2003 -->
<pre>@23:00:49h:   
----------------------------------------------------------- 
bank = C:B0PAGC media0 = 189841.5430153529 media1  
=379630.4183827082 media2 = 1897946.4712205464  
bank = C:B0PAGC sigma0 = 13.375414898406692 sigma1 =9.432699616237176 sigma2 = 4.122992087892885  
bank = C:B0PAGC diffe0 = 3.4162105445632935E7 diffe1 =3.416209933545685E7 diffe2 = 3.4162093089149475E7  
 
button = MEAN (C:LOSTP) STATUS = 0  
button = MEAN (C:LOSTPB) STATUS = 0  
button = RMS (C:LOSTP) STATUS = 0  
button = RMS (C:LOSTPB) STATUS = 0  
button = T:L1COLI STATUS = 0  
button = C:B0PAGC STATUS = 3  
button = RMS (T:RFSUM) STATUS = 0  
button = RMS (T:RFSUMA) STATUS = 0  
button = C:B0ILUM STATUS = 0  
button = SVX DANGER STATUS = 3  
--------------------------------------------------------- 
@23:06:51h: 
--------------------------------------------------------- 
bank = C:B0PAGC media0 = 50.2177179230584 media1 = 49.968302472432455 media2 = 51.3350944519043  
bank = C:B0PAGC sigma0 = 0.1876276507576286 sigma1 = 0.17954594697387324 sigma2 = 0.1686357059185174  
bank = C:B0PAGC diffe0 = 2.735485076904297 diffe1 = 10.266143798828125 diffe2 = 7.700889587402344  
 
utton = MEAN (C:LOSTP) STATUS = 0  
button = MEAN (C:LOSTPB) STATUS = 0  
button = RMS (C:LOSTP) STATUS = 0  
button = RMS (C:LOSTPB) STATUS = 0  
button = T:L1COLI STATUS = 0  
button = C:B0PAGC STATUS = 0  
button = RMS (T:RFSUM) STATUS = 0  
button = RMS (T:RFSUMA) STATUS = 0  
button = C:B0ILUM STATUS = 0  
button = SVX DANGER STATUS = 0  
</pre>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 23;51;40 comment by...rainer -->
from TEVMon log files ...
<!-- Date: Wed Jun 11 23:50:22 2003 -->
<center><a href='/elog/notebooks/silicon/Wed_Jun_11_2003_234421.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2180' ></a><a href='/elog/notebooks/silicon/Wed_Jun_11_2003_234432.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2182' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Wed Jun 11 23;50;51 comment by...rainer -->
<pre>TEVMon Status @23:15h.</pre>



<!-- Date: Thu Jun 12 00:01:20 2003 -->
<center><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_000056.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2187' ></a><a href='/elog/notebooks/silicon/Wed_Jun_11_2003_235906.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2184' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 00;03;14 comment by...rainer -->
<pre>SB2W9L2 - the LAZARUS ladder. is back now, while we mucked around with beam abort gap losses. maybe, after all, it LIKES irradiation a bit. Steve says that it is a grassy ladder so it moght just have to shake off some noise after power up. </pre>


<!-- Date: Thu Jun 12 01:09:11 2003 -->
Since we accidentially left out two wedges SB0W4/5 (<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=19:47:47'> 
here </a>), and interpreted 
their pinky state erronenously as having something to do with being powered off, the trip levels David adjusted (cf. <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=18:36:40'> 
here, last entry </a> need to be readjusted. <p>Since the PDB did not appear to be up to date, it remained just a wild guess which were the trip levels before David re-adjusted them. My guesses are coming from diffing  <i> lookup.txt_before_revert_rw </i> 
 with  <i> lookup.txt_alleged_version_before_david </i> and are subsumed in the current lookup.txt. I refrained from putting all these into the PDB since it probably would make things worse. I take it that whitespace does not play a role in this file because otherwise .... <p> 
IMON reloaded with new lookup.txt. Old one (David's) is <i> lookup.txt_before_revert_rw </i>  
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 01;25;25 comment by...rainer -->
<pre>IMON currents w/o pinkies for quite a while ...</pre>


<!-- Date: Thu Jun 12 06:02:33 2003 -->
<pre>I got paged at around 3:21am because of TEVMon alarms. I reconstruct as follows: 
 
12-Jun-03 03:02:46  
bank = C:LOSTP media0 = 10216.354424753474 media1 = 10184.271090609662 media2 = 10540.104432932101  
bank = C:LOSTP sigma0 = 0.14015137897222127 sigma1 = 0.1482736584992875 sigma2 = 0.2117061278093924  
bank = C:LOSTP diffe0 = 9775.000246937278 diffe1 = 9325.000235569321 diffe2 = 9575.000241884856  
button = SVX DANGER STATUS = 2  
========================================================= 
12-Jun-03 03:03:22 
bank = C:LOSTP media0 = 10243.437758770993 media1 = 10230.937758455228 media2 = 9987.326641189982  
bank = C:LOSTP sigma0 = 0.14436676450596286 sigma1 = 0.1591955661528815 sigma2 = 0.15555117564507548  
bank = C:LOSTP diffe0 = 1925.0000486295903 diffe1 = 2950.00007452327 diffe2 = 1875.000047366486  
button = SVX DANGER STATUS = 0  
======================================================== 
12-Jun-03 03:14:13  
bank = C:LOSTP media0 = 8578.576605601957 media1 = 8554.271049432398 media2 = 9028.993283647113  
bank = C:LOSTP sigma0 = 0.21114391060197926 sigma1 = 0.2830700692977645 sigma2 = 0.568199800731394  
bank = C:LOSTP diffe0 = 20400.000515347354 diffe1 = 21850.000551977442 diffe2 = 22425.000566503168  
button = RMS (T:RFSUM) STATUS = 2  
button = RMS (T:RFSUMA) STATUS = 2  
button = SVX DANGER STATUS = 2  
============================================================= 
12-Jun-03 03:14:37  
bank = C:LOSTP media0 = 9089.409951840029 media1 = 9683.437744624212 media2 = 15074.826769711046  
bank = C:LOSTP sigma0 = 0.3268057433473272 sigma1 = 0.4171153613474501 sigma2 = 0.431874084907703  
bank = C:LOSTP diffe0 = 3800.000095996076 diffe1 = 6400.000161677601 diffe2 = 6375.000161046052  
button = RMS (C:LOSTP) STATUS = 3  
button = RMS (T:RFSUM) STATUS = 2  
button = RMS (T:RFSUMA) STATUS = 2  
button = SVX DANGER STATUS = 3  
 
=&gt; LOSTP TeVMON error state 
 
I hear about loss of RF control and instruct Monitoring Ace to  have a look at the Abort GAP losses in addition to the usual 
plots which he should closely monitor as well.  
</pre>
<!-- Author: rainer -->


<!-- Date: Thu Jun 12 06:06:13 2003 -->
<pre>At the same time, DAQ Ace reports a trip of IB3W0L3 showing on the trigger inhibit but &lt;b&gt; no &lt;/b&gt; trip reported on the gui!! 
IFIX has ONE bar of ISL down. I make the ACE power cycle the  
ladder, to no avail. I find it very strange that the GUI is happy but the trigger inhibit is not. IFIX and GUI communication is ok. I make the ace page the interlock expert (Eva H.) and drive to the control room.</pre>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 09;32;44 comment by...Eva H. -->
<pre>This is why I had the ace do the power cycle.  This bar only came back when Rainer removed IB3W0L3 and the run was recovered.</pre>
<!-- Comment: Thu Jun 12 10;21;07 comment by...Eva H. -->
<pre>Just found out from Andy that the bar going away is normal, since it was showing the minimum value of all the voltages, which obviously was zero due to IB3W0L3 VBias1.  The problem with this guy is quite likely a bad PS.
</pre>
<!-- Comment: Thu Jun 12 10;29;21 comment by...Jason -->
I think that, in general, one should stop the run and start a 
new run (through ColdStart) after changing the hardware database.
<BR>
The ladder configuration is derived from the hardware database at the beginning of the run. If it's changed halfway through the run, you get junk data from that (now unbiased) ladder without knowing it.
<BR>
In this particular case, it doesn't matter since the 
<A HREF="http://www-cdfonline.fnal.gov/internal/ops/svxii/biasInfo/biased_ladders_164555.txt">biasInfo file</A> will show that ladder as being unbiased, and it will be treated as missing from the entire run.
<!-- Date: Thu Jun 12 06:15:45 2003 -->
Eva is on the phone on arrival, and together we check out what's going on. She had made the ACE powercycle wedges and the whole ISL, but to avail. the log file does not show any kind of trip. After a while I notice that on IB3W0L3 Vbias1 is 0 Volts after Eva H. triggers me towards reviewing the trigger inhibit thresholds. So Vbias1 is off and that's why we get the trigger inhibit. On the other hand Vbias is ON so that's why we don't get a trip. I take IB3W0L3 out of the hardware database and reload the CAEN volts file. there was a similar incident  
<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=128&frame=2&anchor=12:24:38'> 
here. </a>  We are trying to make sense out of <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=97&frame=2&anchor=15:23:42'> 
this </a>  to find out more about the nature of the inhibit but since we had the Ace recover and run, the PSmon entries are updated and there is no history available. Eva will look into this further later today. 
  
<!-- Author: rainer -->


<!-- Date: Thu Jun 12 06:27:53 2003 -->
<pre>IMON dies on me again as usual so the final word will have DS current monitor I guess but here is what the IMON log file 
says about bias0 and bias1 
 
1863  0 -32765    84.70    16.00   1864  0 -32765    84.30    18.00  
1863  0 -32765    84.70    16.00   1864  0 -32765    84.30    18.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    35.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    37.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    38.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    37.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    38.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    37.00  
1863  0 -32765    84.70    17.00   1864  1      3     0.00    26.00  
1863  0 -32765    84.70    16.00   1864  1      3     0.00    10.00  
1863  0 -32765    84.70    17.00   1864  0 -16381     0.00    20.00  
1863  0 -32765    84.70    16.00   1864  0 -16381     0.00    24.00  
1863  0 -32765    84.70    17.00   1864  0 -16381     0.00    28.00  
1863  0 -32765    84.70    16.00   1864  0 -16381     0.00    33.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    35.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    36.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    37.00  
1863  0 -32765    84.70    16.00   1864  1      3     0.00    10.00  
1863  0 -32765    84.70    17.00   1864  0 -16381     0.00    23.00  
1863  0 -32765    84.70    16.00   1864  0 -16381     0.00    25.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    26.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    27.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    28.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    29.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    30.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    31.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    32.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    33.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    34.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    35.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    36.00  
1863  0 -32765    84.70    16.00   1864  0 -32765     0.00    35.00  
1863  0 -32765    84.70    17.00   1864  0 -32765     0.00    36.00  
1863  0 -32765    84.70    16.00   1864  1      3     0.00    18.00  
1863  0 -32765    84.70    17.00   1864  1      3     0.00    10.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  0 -32765    84.70    17.00  
1863  0 -32765    84.70    16.00  
1863  1      3    49.40     5.00  
1863  1      3     0.00     0.00  
1863  0 -16381     9.80     0.00  
1863  0 -16381    24.10     3.00  
1863  0 -16381    42.00     7.00  
1863  0 -16381    60.00    10.00  
1863  0 -16381    74.30    14.00  
1863  0 -32765    84.50    14.00  
1863  0 -32765    84.70    14.00  
1863  0 -32765    84.70    15.00  
1863  0 -32765    84.70    16.00  
1863  1      3     0.00     0.00  
1863  0 -16381    53.10     2.00  
1863  0 -16381    61.60     3.00  
1863  0 -32765    77.90     5.00  
1863  0 -32765    84.70     5.00  
1863  0 -32765    84.70     6.00  
1863  0 -32765    84.70     7.00  
1863  0 -32765    84.70     8.00  
1863  0 -32765    84.70     9.00  
1863  0 -32765    84.70    10.00  
1863  0 -32765    84.70    11.00  
1863  1      3    16.30     0.00  
1863  1      3     0.00     0.00  
New time 21863 Thu Jun 12 06:04:23 2003</pre>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 06;28;55 comment by...rainer -->
note that I pasted them next to each other and the events are not necessarily aligned.


<!-- Date: Thu Jun 12 06:45:06 2003 -->
<pre>The web is gone. It occurs to me: do we have TEVMon then ? I page Steve and he concludes as long as we have ACNET plots, we are fine.</pre>
<!-- Author: rainer -->


<!-- Date: Thu Jun 12 06:46:58 2003 -->
at some point, IB4W0L4 was showering DAQ with stuck cell id/stuck ladder HRRs, see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=487&frame=2&anchor=02:36:51.825'> 
here. </a> Indeed I get a lot of stuck ladder errors in the error logger, but I see both z and phi sides, although something is going on in the last chip, see the following plots. AVDD and DVDD currents appear to be normal.
<!-- Author: rainer -->


<!-- Date: Thu Jun 12 06:56:14 2003 -->
<center><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_064859.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2191' ></a><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_064917.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2193' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 06;57;14 comment by...rainer -->
<pre>occupancy plot for IB4W0L4. need to click it to see rise in last chip.</pre>

<!-- Date: Thu Jun 12 07:21:53 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: IB3W0L3 
<li>problem n: 1061 
<li>Status: needs_study_time
<li>Type: BIAS
<li>Action: Incident
<li>entered by: rainer 
<li>description is: dropped VBIAS1<a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=06:06:13'> see here </a>  
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B3W0&layer=L3">report</a></ul>
<!-- Author: Problem DB -->







<!-- Date: Thu Jun 12 11:20:02 2003 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2194&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2195' ></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2196&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2197' ></a></center>
<!-- Author: SCN -->
<!-- Comment: Thu Jun 12 11;27;13 comment by...SCN, Lester -->
<h3>Two SRC fixes</h3>
Didn't get these into the log yesterday.  <ul>
<li> Left: Close up of when BXING appears on the Fib backplane, 
relative to DIGITIZE (FCMD 6) and READOUT (FCMD 5).  Apparently
the BXING has to come a little earlier than DIGITIZE.  See<a href=
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=13:54:38'>
comparison picture</a> 
<li> Right: L1A L2A with some time in between.  What you
see is a L1A, followed by a special L2A 8.6 uS later (what we trigger on) upon which the SRC sends the READ Commands to the VRB
and the BXING and (not shown) DIGITIZE and READOUT to the FIB, and then (10 uS later) a L1DONE.  The real L2A is queued up in
the TS, and doesn't come to the SRC until about 80 uS, upon which
the SRC sends the SCAN commands to the VRB and the Event number as well.
</ul>
Thus, we think the SRC in tracer mode is behaving how it should behave.


<!-- Date: Thu Jun 12 11:41:04 2003 -->
Reset ghostbuster board registers in b0svx02 (<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=155&frame=2&anchor=10:45:46'>Register 0x4800001c  Value 0x21 Register 0x48000020  Value 0xa</a>).  We should make an effort to document the timing structure before 
the 2 SRCs so we can compare to after and see how much safer we are.
<!-- Author: SCN -->


<!-- Date: Thu Jun 12 15:58:04 2003 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2198&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2198  width='300' align='top'></td><td>SVT plot looks ok for store 2676</a></td></tr></table>
<!-- Author: SCN -->


<!-- Date: Thu Jun 12 17:01:29 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Thu_Jun_12_2003_170110.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2200  align='top'></td><td>SB2W0L3 many pipeline out of syncs according to error logger</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 17;40;43 comment by...rainer -->
<pre>Some consecutive events from recent a stream ....

-------------------------------------------------------------
e203 0cfc 991f 101f 7c1f 7d1f 7e1b 981f 971f 1717 1822
1910 1a1b 961f 951f 551f 5616 571f 5816
5919 941f b31f b21f 5819 5918 5a2c 5b1e
b11f b01f 7f1d c2c2 0000 

e204 0cfc ....
-------------------------------------------------------------
e203 47fe 9934 101f 7c18 7d10 7e1b 9834
9734 751b 7616 7758 782a 7919 9634 5c18
5d23 5e26 5f1a 9534 571f 5813 591e 9434
b334 1b1e 1c10 1d14 1e15 1f10 301f b234
751b 7618 771b 785f 7911 7a1b b134 b034
7f1f c2c2 

e204 47fe .....
--------------------------------------------------------------
e203 15fc 9915 101f 191d
1a10 1b2e 1cd0 1d69 1e1c 1f15 301f 9815
531a 5410 559d 56e5 572b 5816 9715 331c
3416 3513 361a 9615 1e18 1f18 1055 11e5
121f 13ec 1452 151f 1628 171f 9515 571f
5811 591d 9415 b315 1118 1216 13a5 14a9
1516 161e 1f1a 3014 3111 321d b215 111f
12e5 1310 141f b115 3c1b 3d13 3e23 3fe5
502b 5115 521b b015 7f1f c2c2 0000 

e204 15fc ....

</pre>


<!-- Comment: Thu Jun 12 18;00;21 comment by...rainer -->
i.e. chip id bit errors which I believe was observed before 
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=136&frame=2&anchor=12:12:44'>
before </a>  according to the <a href= 
'http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W0&layer=L3'> PDB </a>
<!-- Date: Thu Jun 12 17:42:03 2003 -->
<center><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_174133.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2202' ></a><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_174151.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2204' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 17;42;35 comment by...rainer -->
<pre>which produces crappy hit maps like above ...</pre>



<!-- Comment: Fri Jun 13 11;26;36 comment by...rainer, steve -->
bit error 4 did show up in run 164625, see <a href='http://nkek15.fnal.gov/cgi-bin/errmon/makeladder.pl?beginrun=30_days_ago&endrun=today&ladder=S203'> bit reporter </a>
. Steve suspects relation to VRT <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=20:51:34'>
swap </a>.
<!-- Date: Thu Jun 12 18:04:31 2003 -->
<pre>SB4W7L4 reports persistent pinky on DVDD high.</pre>
<!-- Author: rainer  -->


<!-- Date: Thu Jun 12 18:08:26 2003 -->
<center><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_180523.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2206' ></a><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_180554.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2208' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 18;18;10 comment by...rainer -->
last chip a0 on z side freaked aparently known in <a href='http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B4W7&layer=L4'> PDB</a>  


<!-- Comment: Thu Jun 12 18;20;31 comment by...rainer -->
<pre>e474
0cfc 8d0f 00f3 8c0f 8b0f 8a0f 890f 880f
870f a60f 73f3 74f0 75f3 a50f a40f a30f
a20f a10f a217 002f 0198 0230 032e 0433
0530 0631 0730 0831 0930 0a2e 0b33 0c30
0d34 0e31 0f34 102e 112c 1231 1333 1432
1533 1632 172e 1831 1930 1a30 1b2e 1c33
1d31 1e31 1f33 2031 212f 2230 2330 2431
252f 2632 272e 2832 2932 2a31 2b2c 2c31
2d33 2e31 2f2f 3033 3130 3231 332d 3431
3531 3630 372f 3833 3931 3a32 3b30 3c33
3d2e 3e32 3f30 402f 4131 4230 432e 4431
4531 4634 4730 482e 4933 4a2f 4b33 4c2e
4d30 4e2f 4f33 502f 5132 5236 5330 5431
552e 5631 5733 5833 5931 5a2e 5b2d 5c2e
5d2f 5e2f 5f2e 602b 6131 6230 6330 642e
6531 662f 6730 6832 6932 6a2f 6b30 6c2e
6d31 6e30 6f2b 7030 7130 7230 7330 742e
7531 762d 7733 782f 792e 7a30 7b2e 7c2f
7d33 7e30 7f30 c2c2 0000 

e480 0cfc  ...</pre>


<!-- Date: Thu Jun 12 18:27:45 2003 -->
<center><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_180802.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2210' ></a><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_182732.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2213' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 18;31;08 comment by...rainer -->
<pre>SB4W7L4 DVDD (left) and AVDD (right) currents ... switch on occured 15:25 according to PSGui.</pre>


<!-- Date: Thu Jun 12 18:56:39 2003 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2216&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2216' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2217&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2217' width='300'></a></center>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 18;59;14 comment by...rainer -->
 DVDD current of SB4W7L4: left short history, right long history. Dave Stuart Day 880 is the day we got online after a 3 day shutdown plus thermo cycle. Comparing with 
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=18:27:45'>
left </a>  above plot, it is clear we need to adjust the limits. <p> This is was power and thermo cycling does for you ....



<!-- Date: Thu Jun 12 19:12:48 2003 -->
<center><a href='/elog/notebooks/silicon/Thu_Jun_12_2003_191234.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2219' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 19;16;01 comment by...rainer -->
This is the plot I would have liked to see a looooooooong time  ago.<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=18:34:42'>
(here  </a>  and <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=16:12:12'>
here </a>). Could we not have this 'same day' function ? <p>

this is what powering off does to you ....


<!-- Date: Thu Jun 12 19:24:29 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W7L4 
<li>problem n: 1062 
<li>Status: needs_observation
<li>Type: DVDD
<li>Action: IMON_limits
<li>entered by: rainer 
<li>description is: Adjusted 1206 DVDD 130 10 100 170 -> 195  10  165  215  to account for new limits after access (see  <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=18:56:39'>  here </a>).  
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B4W7&layer=L4">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Thu Jun 12 19:44:55 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W0L3 
<li>problem n: 829 
<li>Status: needs_study_time
<li>Type: Chip_bit_error
<li>Action: Incident
<li>entered by: rainer 
<li>description is: bit4 seems to be stuck high <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=17:01:29'> E log </a>   
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W0&layer=L3">report</a></ul>
<!-- Author: Problem DB -->
<!-- Comment: Thu Jun 12 19;45;40 comment by...rainer -->
<pre>may some of the local bit gurus please verify ... my bitology is not so great  any more after this last night.</pre>


<!-- Date: Thu Jun 12 20:55:14 2003 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2222&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2222' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2223&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2223' width='300'></a></center>
<!-- Author: rossin -->


<!-- Date: Thu Jun 12 20:56:36 2003 -->
<pre>the two plot on the previous esntry were: 
before HRR (left) 
after HRR (right) 
but still the situation is unstable.</pre>
<!-- Author: rossin -->


<!-- Date: Thu Jun 12 22:30:57 2003 -->
<pre>Observed pinky SB4W7L4 current is 130mA limit 165-215</pre>
<!-- Author: Alberto, Rainer -->
<!-- Comment: Thu Jun 12 22;32;04 comment by...rainer  -->
<pre>on DVDD</pre>
<!-- Date: Thu Jun 12 22:31:40 2003 -->
<pre>MCR lose beam due to quench at D0. Shift crew performs post quench checkout.</pre>
<!-- Author: rainer -->



<!-- Date: Thu Jun 12 23:06:23 2003 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2224&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2224  width='300' align='top'></td><td>trying to evaluate error sources</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 23;07;09 comment by...rainer -->
generation of html as promised in the menu does not work (freezes netscape ...)



<!-- Comment: Thu Jun 12 23;48;36 comment by...rainer -->
<pre>no it just takes forever ....</pre>
<!-- Date: Thu Jun 12 23:19:22 2003 -->
<pre>Top Ten of error throwing ladders in SVXError logger: 
SB2W4L4  C1 
SB3W6L4  C8 
SB1W10L4 C5 
SB1W3L4  C5 
SB0W3L4  C5 
SB0W3L4  C0-6 
SB2W0L0  C1 
SB0W3L4  C1 
SB0W3L4  C2</pre>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 23;49;09 comment by...Rainer -->
<pre>Base of this is Run #164625</pre>
<!-- Date: Thu Jun 12 23:30:32 2003 -->
<pre>The reds CO Roberto did not like in SVXMon: 
 
SB2W0L3 
        Bit Error (not 0/1)  This error was present in  6371 events out of  6371 
        C2 Termination       This error was present in  4071 events out of  6371 
        Unknown Channel      This error was present in  2300 events out of  6371 
</pre>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 23;32;28 comment by...rainer -->
<a href=
'http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W0&layer=L3'> PDB</a> 



<!-- Date: Thu Jun 12 23:34:39 2003 -->
<pre>SB4W6L3 
        Bit Error (not 0/1)  This error was present in  6371 events out of  6371 
        C2 Termination       This error was present in  3443 events out of  6371 
        Unknown Channel      This error was present in  2928 events out of  6371 
 </pre>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 23;37;11 comment by...rainer -->
<a href=
'http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B4W6&layer=L3'> PDB </a>



<!-- Date: Thu Jun 12 23:40:04 2003 -->
<pre>SB2W7L1 
        Bit Error (not 0/1)  This error was present in  6371 events out of  6371 
        C2 Termination       This error was present in  3443 events out of  6371 
        Unknown Channel      This error was present in  2928 events out of  6371 
</pre>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 23;41;39 comment by...rainer -->
<pre>&lt;a href='http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W7&layer=L1'&gt; PDB &lt;/a&gt;</pre>

<!-- Comment: Thu Jun 12 23;43;04 comment by...rainer -->
<a href='http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W7&layer=L1'> PDB </a>




<!-- Date: Thu Jun 12 23:45:20 2003 -->
<pre>SB2W0L0 
        Bit 1 Low            This error was present in  4317 events out of  6371 
        Bit Error (not 0/1)  This error was present in  492 events out of  6371 
        C2 Termination       This error was present in  13 events out of  6371 
        Drop Readout (d0 ch) This error was present in  20 events out of  6371 
        Unknown Chip Error   This error was present in  4065 events out of  6371 
        ff or 7f Error       This error was present in  992 events out of  6371 
</pre>
<!-- Author: rainer -->
<!-- Comment: Thu Jun 12 23;47;42 comment by...rainer -->
<a href=
'http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W0&layer=L0'> PDB </a>




<!-- Date: Thu Jun 12 23:54:01 2003 -->
<pre>I'll keep the above links in order to verify the status of the problem data base. However, I prefer to wait until DS Tool output is available, i.e. tomorrow morning (hopefully).</pre>
<!-- Author: rainer -->


<!-- Date: Fri Jun 13 00:03:26 2003 -->
<center><a href='/elog/notebooks/silicon/Fri_Jun_13_2003_000207.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2226' ></a><a href='/elog/notebooks/silicon/Fri_Jun_13_2003_000234.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2228' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Fri Jun 13 00;08;01 comment by...rainer -->
After turn off the HV, there is a big step in the DVDD current,
and hence  SB4W7L4 DVDD turns pink. Was this the case before ?
DS current plotter will hopefully tell tomorrow. I keep the
newly tuned (<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=22:30:57'>
here </a> and 
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=19:24:29'>
here </a>) current limits for the time being. 



<!-- Comment: Fri Jun 13 11;10;36 comment by...rainer -->
Well, sometimes i don't think straight any more. i have the dvdd currents already plotted <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=18:56:39'>
before </a> 
<!-- Date: Fri Jun 13 08:40:02 2003 -->
<pre>New day, new cup of tea. Will try to work on IB3W0L3 CAEN power HV power supply. First, I'll get it back into the hardware database and see whether I read some voltage and current on STANDBY.</pre>
<!-- Author: rainer -->

<!-- Date: Fri Jun 13 10:21:21 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: IB3W0L3 
<li>problem n: 1061 
<li>Status: needs_observation
<li>Type: BIAS
<li>Action: Diagnosis
<li>entered by: rainer 
<li>description is: ladder magically shows again bias1 voltage in standby. needs to be verified for full hv during access. watch out it dropping vbias1 will produce trigger inhibit but no trip will show up in gui. 
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B3W0&layer=L3">report</a></ul>
<!-- Author: Problem DB -->
<!-- Comment: Fri Jun 13 10;23;13 comment by...rainer, steve -->
<pre>We contacted matt on this. they have no suitable spare 
worth swapping it with. if the trip on vbias1 comes back and
they have a chance to go in, they would like to pull the hv cable to see whether the trip is real but most likely adc 
trouble. he was not surprised to hear it come back after being
turned off.</pre>

<!-- Date: Fri Jun 13 10:31:32 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W7L4 
<li>problem n: 861 
<li>Status: needs_study_time
<li>Type: Chip_other
<li>Action: Diagnosis_update
<li>entered by: nahn 
<li>description is: This turns out to be just for the last chip.   Rainer noticed <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=18:56:39'> a current change</a> as well coincident with the 3 day power cycle when the bit errors occured.  In the <A href=' http://nucsb03.fnal.gov/~stuart/errors/ir027f77.0001cali_e474.html.gz'>first DPS ON calib run</a>  the last chip has <ol> <li> Chip ID 0xa2 instead of 0xa0 (thus c2c2) <li> Appears to be DPS OFF (in a DPS ON calib run) <li> Has the wrong cap ID (perhaps has the wrong Pipe Depth) </ol>  In <a href='http://nucsb03.fnal.gov/~stuart/errors/ar0281b2.0001_e474.html.gz'>another recent run</a> it shows some events DPS ON (but still Read All, and wrong Chip ID) and other events DPS OFF.  This suggests that we cannot put this chip in the correct running mode, even though the init readback succeeds.  We will investigate with SVXDAQ and perhaps power cycle to see if it he!
lps this last chip.  Worst case is to chop at CHIP ID 0x21.  
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B4W7&layer=L4">report</a></ul>
<!-- Author: Problem DB -->





<!-- Date: Fri Jun 13 16:36:55 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Fri_Jun_13_2003_163534.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2230  align='top'></td><td>SB4W7L4 last Chip doesn't go into DPS OFF even after power cycling.  </a></td></tr></table>
<!-- Author: SCN, Rainer, David -->
<!-- Comment: Fri Jun 13 16;44;52 comment by...SCN, Rainer, David -->
I meant that this Chip will not go into DPS ON (while the
rest of the ladder does) We will check for physics from this
chip, if there is not, we will chop this chip.


<!-- Date: Fri Jun 13 16:46:42 2003 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2231&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2232  width='300' align='top'></td><td>No Hits in last Chip (We suspect the pipeDepth is  not constant during the run...)</a></td></tr></table>
<!-- Author: SCN, David, Rainer -->


<!-- Date: Fri Jun 13 19:46:40 2003 -->
Need to initiate cold start via RC after clock will be switched 
from free to normal mode to get chips out of potential high current states. 
 
Instructions how to switch the clock from free to normal mode found 
<a href='http://www-cdfonline.fnal.gov/ops/ace2help/Clock/Using_the_CDF_Clock.html#Mode'>here</a>. 
  
<!-- Author: rainer -->
<!-- Comment: Sat Jun 14 07;31;10 comment by...SCN -->
Just a reminder...<BR>
Noone should ever touch the Clock without explicit instructions
from the Ops Manager (which Rainer had, in this case).  
<!-- Date: Sat Jun 14 04:11:30 2003 -->
I come in to change the clock. I exercise one more time 
a cold start to see everything is ok. 
 
Then I make the ace stop the run, and switch to normal clock running. I initiate a cold start but however, all fib I get partition zero clock error from run control and the PCC shows sync timing error. Silicon crates trip, foghorns blast and I turn silicon to off to be sure via IFIX and page Steve. He will  
 
I realize that in the clockGUI SetMClockDelay is set to 0, but the clock shift instructions say it should be B. With Steve on the phone, he suggests clearing the errors  on the PCC but the one error 'Sync Timing' LED stays on and he askes for the status of the LED numbers with show 0 0. We try setting it to B but it still reports 0. He comes in.
<!-- Author: rainer -->

<!-- Date: Sat Jun 14 04:58:52 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W7L4 
<li>problem n: 861 
<li>Status: needs_observation
<li>Type: Chip_other
<li>Action: Change_last_chip
<li>entered by: nahn 
<li>description is: We chopped the last chip, set the ISEL and Rdriver for that Chip to 0, and its threshold to 255.  However, if it truly does not obey its shadow parameter settings, the Chip parameters may or may not do anything. We also adjusted the lower IMON limit back to 130.  The upper limit awaits data taking... 
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B4W7&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Sat Jun 14 05:05:14 2003 -->
<pre>Silicon brought back to standby via IFIX. SVX B5 W3 does not come up immediately but powering from the GUI fixes it. No dead bodies on the ground. Uff.</pre>
<!-- Author: rainer, steve -->



<!-- Date: Sat Jun 14 07:37:10 2003 -->
<h3>Belated Access Report</h3> 
Access was on 6/13 at around 2 PM. 
David and I went in to investigate a bit 4 error in SB0W2L3.  
The error disappeared when we swapped HDIs 3 and 4, and did not 
reappear when we swapped them back.  Since the error was only seen 
in one run, we left it in working condition with no swaps etc.<P> 
We also attempted to troubleshoot the non-functional reset lines 
for CAEN crates 15 and 16.  The configuration is that a spare 
ethernet line has been converted into a reset line, and there 
are two lemo cables spliced into the ethernet cable, labeled #3 and #4.  One of the lemo cables (#4 when we started) goes into a T which goes into crate 15, and also goes into another lemo cable which goes into crate 16.  We attempted a reset using both #3 and #4, did not see any evidence that a crate was reset. Then we swapped the T, but the one we took out shows no evidence of being broken, so I expect this did not help (but also did not hurt).
<!-- Author: SCN, Rainer, David -->
<!-- Comment: Sat Jun 14 11;39;37 comment by...rainer -->
access was <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=11:31:01'>
success </a>  for SB0W2L3.
<!-- Date: Sat Jun 14 07:49:20 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W0L3 
<li>problem n: 829 
<li>Status: needs_observation
<li>Type: Chip_bit_error
<li>Action: Diagnosis
<li>entered by: nahn 
<li>description is: Bit 4 confirmed during studies time, in Access move cables to isolate ladder or DAQ, and problem vanished.  Moved back, problem stayed gone. David added some slack to the DOIM cable going in, maybe that helped. 
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W0&layer=L3">report</a></ul>
<!-- Author: Problem DB -->
<!-- Comment: Sat Jun 14 09;12;14 comment by...rainer -->
<pre>Have Bit Error (not 0/1) in SVXMon
This problem persists for 2049/2223 consecutive monitor events .
Does not look like it is gone.</pre>
<!-- Comment: Sat Jun 14 12;56;49 comment by...rainer -->
<pre>...but the strip oocupancy map looks good and data is coming, so I guess ignore the above comment and see below....</pre>
<!-- Date: Sat Jun 14 08:58:36 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Sat_Jun_14_2003_085828.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2234  align='top'></td><td>le 'du jour' ....</a></td></tr></table>
<!-- Author: rainer -->


<!-- Date: Sat Jun 14 08:59:44 2003 -->
<center><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_085904.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2236' ></a><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_085932.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2238' ></a></center>
<!-- Author: Rainer -->
<!-- Comment: Sat Jun 14 09;09;21 comment by...rainer -->
SVXMon du jour of Run 164725. There is SB5W0L3 not yielding any data in this run. was fine in proceeding run 164625.
<a href='http://nkek15.fnal.gov/cgi-bin/errmon/makeladder.pl?beginrun=30_days_ago&endrun=today&ladder=S503'> Bitreporter </a>.

<a href='http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B5W0&layer=L3'> PDB </a>.



<!-- Date: Sat Jun 14 09:13:49 2003 -->
Asking for HRR  and watching ...
<!-- Author: rainer -->


<!-- Date: Sat Jun 14 10:38:44 2003 -->
<pre>got paged because X session on b0dap42 died. I instruct Ricardo E. how to restart PSGui. Now it runs on b0dap42 instead of b0dap44. </pre>
<!-- Author: rainer -->
<!-- Comment: Sat Jun 14 10;49;38 comment by...rainer -->
<pre>I guess we are running the old version of the gui now, not yimei's one. Don't want to muck around with it now.  Otherwise
I send again zeroes where they don't belong to.</pre>


<!-- Date: Sat Jun 14 10:55:38 2003 -->
<center><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_105415.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2240' ></a><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_105518.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2242' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Sat Jun 14 11;00;15 comment by...rainer -->
I got a chance to power cycle the ladder SB5W0L3 before run 164727 and coldstart (they were working on something else). Still to no avail. SB5W0L3 tripped on DVDD on coldstart but could be recovered. No more dead top dileptons with my name on them.


<!-- Date: Sat Jun 14 11:06:52 2003 -->
<center><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_110633.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2244' ></a><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_110641.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2246' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Sat Jun 14 11;17;09 comment by...rainer -->
<pre>SB5W0L3 gives some data on chip0/phi-side but pipeline out of sync errors and subsequent chips yield no data.

look at some events:

e503 
73fc 093c 00f3 0083c 7d04 7e09 7ff3 073c 063c 3f01 4018
41f3 053c 043c 20f3 210a 220f 230c 2403
2d03 2e0b 2f09 30f3 4407 450c 460b 470b
480e 4903 7ff3 7f50 7f55 7f75 6f55 7b55
4f51 7f55 7b55 4f51 7b55 7f55 7b55 4f51
7f55 7f55 7b55 4f52 7f55 7f55 7b55 4f51
7f55 7855 7b55 4f52 7f55 7f55 7b55 4f51
7fd0 d0c2 c2c4 0000 

e504 73fc 8d22  ....
---------------------------------------------------------
e503 
44fe 091c 00f3 081c 071c 061c 40f3 4173 4216 4307
051c 041c 7f03 4f52 7fd0 d0c2 c2c4 
e504 44fe 8d12 ....

bit 6 stuck low ....</pre>

<!-- Comment: Sun Jun 15 09;49;45 comment by...SCN -->
<pre>Bit 7 I think...093c becomes 893c   083c becomes 883c, 073c becomes 873c...so you are not getting Chip IDs.</pre>
<!-- Date: Sat Jun 14 11:25:29 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB5W0L3 
<li>problem n: 783 
<li>Status: needs_study_time
<li>Type: DropReadOut_nonRX
<li>Action: Incident
<li>entered by: rainer 
<li>description is: ladder fails in run  <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=08:59:44'> 164725 </a>. Pipeline out of sync on chip0 and no data from the rest of the chips. looks like bit 6 is dropped. <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=11:06:52'> here </a>  
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B5W0&layer=L3">report</a></ul>
<!-- Author: Problem DB -->
<!-- Comment: Sat Jun 14 18;26;30 comment by...rainer -->
<pre>really is bit7 ....</pre>
<!-- Comment: Sat Jun 14 18;31;21 comment by...rainer -->
<pre>we will go in at 9m tonight to diagnose problem and/or swap RX. 
it is e503 in b0fib04 slot 16 Channel 3. The crate is NE top.</pre>
<!-- Date: Sat Jun 14 11:31:01 2003 -->
<center><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_113041.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2248' ></a><a href='/elog/notebooks/silicon/Sat_Jun_14_2003_113050.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2250' ></a></center>
<!-- Author: rainer -->
<!-- Comment: Sat Jun 14 11;32;32 comment by...rainer -->
<pre>resurrexit SB0W2L3 after access.</pre>


<!-- Comment: Tue Jun 17 11;46;59 comment by...David -->
<pre>This probably should have been SB2W0L3, which is still showing bit 4 high.</pre>
<!-- Date: Sat Jun 14 11:55:16 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W7L4 
<li>problem n: 1062 
<li>Status: needs_observation
<li>Type: DVDD
<li>Action: IMON_limits
<li>entered by: rainer 
<li>description is: after chopping last chip upper limit adjusted to old value while data taking(see above). 
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B4W7&layer=L4">report</a></ul>
<!-- Author: Problem DB -->






<!-- Date: Sat Jun 14 22:53:41 2003 -->
<b>Silicon Access report</b> 
<p> 
Swapped DOIM cables on two ladders to isolate errors: 
<ul> 
<li>SB5W0L3: bit 7 low in pretest. Swapped DOIM cable with channel 2 and errors now show up in pipe 2, so problem is internal.</li> 
<li>SB0W4L4: c2c2 termination at beginning of run in svxdaq. Swapped DOIM cable with channel 2 and error showed up in pipe 2, so problem is internal.</li></ul> 
Next step might be to look at the light output of these two ladders.
<!-- Author: Team Clark & Rainer -->
<!-- Comment: Sat Jun 14 23;18;43 comment by...Clark, Rainer -->
<br>Curious!? Initialization consistently failed during the pretest and post-test of SB0W4L2 while in the collision hall. There is no history of this in the PDB and it is not masked. After exiting the hall we ran a COLDSTART with run control as usual and this ladder passed with no problems. 


<!-- Comment: Sun Jun 15 09;09;30 comment by...rainer -->
<pre>I remember when switching L2 with L4, now SB0W4L4 has the init read back masked so maybe that's why it works. it was strange that sometimes we could read back correctly, sometimes not.</pre>
<!-- Date: Sun Jun 15 08:59:55 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Sun_Jun_15_2003_085929.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2252  align='top'></td><td>svt du jour</a></td></tr></table>
<!-- Author: rainer -->
<!-- Comment: Sun Jun 15 09;01;18 comment by...rainer -->
<pre>ooops .... too early</pre>
<!-- Date: Sun Jun 15 08:59:55 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Sun_Jun_15_2003_085929.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2252  align='top'></td><td>svt du jour</a></td></tr></table>
<!-- Author: rainer -->





<!-- Date: Sun Jun 15 11:24:13 2003 -->
Apparent clock RF glitch to CDF, in conjunction with beam backgrounds shooting up, see main  
<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=497&frame=2&anchor=10:58:03.901'> 
Elog.</a> PCC board shows several errors   (RF missing, RF sync. etc) and RC complains about clock problems. There are several trips on DVDD. I tell the aces to switch off the silicon to avoid high current states in the absence of clock. Luckily, Lester 
happens to be here and he knows how to switch off faster by killing the interlocks. A rack trips but seems to reset itself.  
I page Steve to verify decision and he says it's ok. 
The silicon is now off and we wait until clock experts get the clock back up before switchting the silicon back on.  
  
<!-- Author: Rainer -->
<!-- Comment: Sun Jun 15 11;28;14 comment by...Rainer -->
<pre>Here the sequence of events:

Sun Jun 15 10:37:55 CDT 2003  
  TRIP: L00  WEST  W2 DVDD3 OverCurrent
  TRIP: SVX  B0  W2 DVDD4 OverCurrent
  TRIP: SVX  B1  W4 DVDD4 OverCurrent
  TRIP: SVX  B1  W11 DVDD4 OverCurrent
  TRIP: SVX  B4  W1 DVDD4 OverCurrent
  TRIP: SVX  B4  W2 DVDD4 OverCurrent
  TRIP: SVX  B4  W8 DVDD4 OverCurrent
  TRIP: SVX  B4  W11 DVDD4 OverCurrent
  TRIP: SVX  B5  W1 DVDD4 OverCurrent
  TRIP: SVX  B5  W3 DVDD4 OverCurrent
  TRIP: SVX  B5  W7 DVDD4 OverCurrent
  TRIP: SVX  B5  W8 DVDD4 OverCurrent
  TRIP: SVX  B5  W10 DVDD4 OverCurrent
Sun Jun 15 10:43:12 CDT 2003
  IFix -  success: Silicon to STANDBY  
Sun Jun 15 10:45:15 CDT 2003
  TRIP: ISL  B5  W5 External Trip (and everybody else ...)
</pre>


<!-- Date: Sun Jun 15 11:38:02 2003 -->
<center><a href='/elog/notebooks/silicon/Sun_Jun_15_2003_113718.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2258' ></a><a href='/elog/notebooks/silicon/Sun_Jun_15_2003_113707.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2256' ></a></center>
<!-- Author: Rainer -->
<!-- Comment: Sun Jun 15 11;39;44 comment by...Rainer -->
<pre>TEVMon reports high abort gap losses, have the SciCo call MCR to find out where that comes from. Silicon is off. Apparently
loss of cable TV relates to loss of clock loss.</pre>


<!-- Date: Sun Jun 15 11:58:43 2003 -->
<pre>b0svx06 is found in red state by RC. shift crew reboot it and it comes up ok.</pre>
<!-- Author: Rainer -->


<!-- Date: Sun Jun 15 12:29:24 2003 -->
Since the power/RF glitch I see persistent high abort gap 
rates as shown <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=497&frame=2&anchor=12:13:40.500'> 
here </a>. This is from ACNET so I guess it's real. We won't be able to switch back on with beam conditions like this.  
<!-- Author: Rainer -->
<!-- Comment: Sun Jun 15 12;31;08 comment by...Rainer -->
<pre>Event = 181692 Time:  15-Jun-03 01:53:39        B0PAGC  = 0     MEAN (C:LOSTP)  = 3     52667:RMS (C:LOSTP)  = 3 
Event = 181717 Time:  15-Jun-03 01:54:29        B0PAGC  = 0     MEAN (C:LOSTP)  = 3     52716:RMS (C:LOSTP)  = 3 
Event = 181718 Time:  15-Jun-03 01:54:31        B0PAGC  = 0     MEAN (C:LOSTP)  = 3     52765:RMS (C:LOSTP)  = 3 
Event = 181760 Time:  15-Jun-03 01:55:56        B0PAGC  = 0     MEAN (C:LOSTP)  = 3     52814:RMS (C:LOSTP)  = 2 
Event = 181768 Time:  15-Jun-03 01:56:12        B0PAGC  = 0     MEAN (C:LOSTP)  = 3     52863:RMS (C:LOSTP)  = 0 
Event = 181821 Time:  15-Jun-03 01:57:58        B0PAGC  = 0     MEAN (C:LOSTP)  = 0     52912:RMS (C:LOSTP)  = 0 
Event = 197408 Time:  15-Jun-03 10:40:06        B0PAGC  = 2     MEAN (C:LOSTP)  = 0     52961:RMS (C:LOSTP)  = 0 
Event = 197411 Time:  15-Jun-03 10:40:12        B0PAGC  = 3     MEAN (C:LOSTP)  = 0     53010:RMS (C:LOSTP)  = 0 
Event = 198548 Time:  15-Jun-03 11:18:17        B0PAGC  = 3     MEAN (C:LOSTP)  = 0     53059:RMS (C:LOSTP)  = 0 
Event = 198587 Time:  15-Jun-03 11:19:36        B0PAGC  = 2     MEAN (C:LOSTP)  = 0     53108:RMS (C:LOSTP)  = 0 
Event = 198592 Time:  15-Jun-03 11:19:46        B0PAGC  = 0     MEAN (C:LOSTP)  = 0     53157:RMS (C:LOSTP)  = 0 
Event = 198634 Time:  15-Jun-03 11:21:10        B0PAGC  = 0     MEAN (C:LOSTP)  = 0     53206:RMS (C:LOSTP)  = 0 
Event = 198635 Time:  15-Jun-03 11:21:12        B0PAGC  = 2     MEAN (C:LOSTP)  = 0     53255:RMS (C:LOSTP)  = 0 
Event = 198638 Time:  15-Jun-03 11:21:18        B0PAGC  = 3     MEAN (C:LOSTP)  = 0     53304:RMS (C:LOSTP)  = 0 

and B0PAGC high ever since ...</pre>

<!-- Comment: Sun Jun 15 12;38;17 comment by...Rainer -->
<pre>Rates in TevMON of abort gap losses are about 13-15kHz and persisting. I had SciCo ask MCR one more time and they said
they would do something about it ...</pre>


<!-- Date: Sun Jun 15 16:12:29 2003 -->
Continuation of the RF clock saga: 
 
got paged at 13:22h: clock is back. I am on site and determine  
tevmon backgrounds are good (some weirdness  
<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=497&frame=2&anchor=13:22:00.317'> 
here</a>. <p> Inform SciCo that getting back silicon might take a while so up to him whether to start with a partition w/o us or wait. We both agree to try getting silicon in right away. 
 
I power up SVX B0 W3 and B1 W2  and ISL B1 W0 as guinea pigs 
as well as L00 WEST W0. first two work fine L00 gives handshake error but after second try works fine. Have shift crew power up to standby and on. Silicon LV and HV up by 13:46h.  
<p> 
Ask shift crew to go through cold start. Have trip on SVX B4W9L4 AVDD  but reset and try again. However cold start fails, page Steve. Failure for several reasons:<p> 
 
-b0svx00,01,02,03,05,06,07,08 don't respond and need CPU reset in first level trailer. this necessitates cleanup of event builder. 
<p> 
-b0svx01 ,05 07 and 08 harbor several VRB which needs     resetting.Steve 
 suggests resetting them all since hung-up is likely connected 
  to clock glitch. 
<p>  
Upon Steve's suggestion divorce myself from main run control which in the meantime had died. In my own partition, upon 
cold  start/config, have still VRB's failing but regained control 
over all VRB/b0svx0x crates. another cycle to reset VRB's, and I confess to have forgotten b0svx08 at the first shot.  It's around the corner, I know. 
<p> After another iteration, have all crates reset and shephered and give Silicon back to CDF  around 14:47h. 
<p> 
While I am trying to get a cold start through, ISL temperature  
almost reaches alarm limit (See <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=497&frame=2&anchor=15:02:11.177'> 
here </a>), expert paged, but as I get my cold start through in my own rc, the temperature falls down again. Shift crew will watch it. 
 
 
<!-- Author: Rainer -->
<!-- Comment: Sun Jun 15 16;22;28 comment by...Rainer -->
PS.: Steve Nahn showed up in the most wacky sandals I have ever seen.


<!-- Comment: Sun Jun 15 21;15;54 comment by...Andy H. -->
re: ISL temperatures -- this is typical behavior; initting the chips into their low-current states results in a noticeable drop in temperature for the ISL.  Makes for a nice reminder, actually.
<!-- Comment: Mon Jun 16 00;20;59 comment by...Rainer -->
OK. thanks to Eva H. and Andy.
<!-- Date: Sun Jun 15 17:53:49 2003 -->
<pre>Stuck ladder in IB4W0L4 in run 164775. 
Same thing happen in run #164555 
 
Changed file 
/cdf/people1/bachacou/SvxMonConfig/stuckChips.txt 
 
From 
 
e100 ?2 
e1a4 ?8 
e134 ?8 
e244 ?c 
e590 ?2 
 
to 
e100 ?2 
e1a4 ?8 
e134 ?8 
e244 ?c 
e590 ?2 
f404 ?0 
f404 ?1 
f404 ?2 
f404 ?3 
 
So that f404 chip 0-3 are masked in the stuck Ladder logic in SVXMON. 
 
</pre>
<!-- Author: Rainer & Gino -->
<!-- Comment: Sun Jun 15 21;45;28 comment by...Rainer -->
CO says he had restarted the SVXMon but I have seen at least twice the error  <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2003&action=view&page=498&frame=2&anchor=21:06:42.910'>
again</a>. I will make he CO restart SVXMon again ...
 



<!-- Date: Sun Jun 15 22:12:48 2003 -->
this guy IB4W0L4 is nasty. Continue to get HRRs from C0-3. so 
the masking does not work ?? I suspect a mapping problem, maybe  
because it is ISL. 
<p> main links: 
<a href=' 
http://nkek15.fnal.gov/cgi-bin/errmon/makeladder.pl?beginrun=30_days_ago&endrun=today&ladder=I404'> bitreporter</a> <t>  
   <a href=' 
http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=ISL&wedge=B4W0&layer=L4'> PDB </a>
<!-- Author: Rainer -->


<!-- Date: Sun Jun 15 22:14:30 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Sun_Jun_15_2003_221421.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2260  align='top'></td><td>just for my sanity .... SVT of this store</a></td></tr></table>
<!-- Author: Rainer -->


<!-- Date: Sun Jun 15 23:00:18 2003 -->
<pre>hmmm.... looking at the hardware database chip IDs I see the following for chip ids: 
0 0x8f 
1 0x8e 
2 0x8d 
3 0x8c 
.... 
looking at the remaining entries in the stuckChips.txt file I come to the conclusion that you rather entere chip IDs then chip 
number (0-3 etc.). therefore update and also include old offender  
e034: 
 
new file stuckChips.txt 
e100 ?2 
e1a4 ?8 
e134 ?8 
e244 ?c 
e590 ?2 
f404 ?f 
f404 ?e 
f404 ?d 
f404 ?c 
e034 ?d 
e034 ?c 
e034 ?b 
e034 ?a 
e034 ?9 
e034 ?8 
e034 ?7 
 
hope that solves the problem ....</pre>
<!-- Author: Rainer -->


<!-- Date: Sun Jun 15 23:02:42 2003 -->
<center><a href='/elog/notebooks/silicon/Sun_Jun_15_2003_230118.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2262' ></a><a href='/elog/notebooks/silicon/Sun_Jun_15_2003_230135.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2264' ></a><a href='/elog/notebooks/silicon/Sun_Jun_15_2003_230217.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2266' ></a><a href='/elog/notebooks/silicon/Sun_Jun_15_2003_230226.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2268' ></a></center>
<!-- Author: Rainer -->
<!-- Comment: Sun Jun 15 23;34;22 comment by...Rainer -->
<pre>That's the occupancy plot for IB4W0L4 on z and phi side. If my theory is correct then the offender is the first chip which shows that wing thing. I have nonetheless masked the other chips as well, just to be safe for this store (I need to sleep).
</pre>

<!-- Comment: Sun Jun 15 23;35;05 comment by...rainer -->
<pre>Will revert back to masking first chip only after this store.</pre>


<!-- Date: Sun Jun 15 23:38:10 2003 -->
While working in L2, they need to stop and start the run. Config worked fine, but on Activate they get a BUSY Timeout from b0svx02. I suspect EVB cockup as indicated by the shift instructions but I fail to find the SCPU  error message in the error logger. So poor Steve gets paged again, and he confirms that it is  EVB problem. Advice to abort run and clear EVB. Run starts then fine again.   
<!-- Author: Rainer -->


<!-- Date: Sun Jun 15 23:44:58 2003 -->
In this run 164797 I have  activated the new SVXMon mask file <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=23:00:18'> 
above</a>. Let's keep fingers crossed that this masks the HRRs from IB4W0L4 and SB0W3L4, the known offenders which send HRRs. Need to verify tomorrow with the <a href='http://nkek15.fnal.gov/cgi-bin/errmon/makeladder.pl?beginrun=30_days_ago&endrun=today&ladder=I404'> bitreporter </a>.
<!-- Author: Rainer -->


<!-- Date: Mon Jun 16 00:16:37 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Mon_Jun_16_2003_001632.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2270  align='top'></td><td>strip occupancy of IB4W0L4.</a></td></tr></table>
<!-- Author: Rainer -->


<!-- Date: Mon Jun 16 00:45:16 2003 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W0L0 
<li>problem n: 636 
<li>Status: needs_study_time
<li>Type: Chip_bit_error
<li>Action: Incident
<li>entered by: rainer 
<li>description is: various bit errors reappear since 164762 (power off and clock problems reported   <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=158&frame=2&anchor=11:24:13'> here)</a>   
<li>See <a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B2W0&layer=L0">report</a></ul>
<!-- Author: Problem DB -->
<!-- Comment: Mon Jun 16 01;03;52 comment by...Rainer -->
<pre>Got DRO from SB2W0L0 reported in error logger, Bit0/1 Highff or 7f error, c2c2, unknown chip error, illegal termination. smoe events:

e200 76fe 838d 8080 8286 8389
84a0 85e2 868c 8788 88e6 89e5 8abe 8b99
8c83 c1f0 
e201 76fe ...
------------------------------------------
e200 4dfc 83a0 8080 c9ef 0000 e201 4dfc
-------------------------------------------
e200 88fe 839e 8083 829e a19e a09e fff1 c2c2 0000 
e201 88fe ....</pre>
<!-- Date: Mon Jun 16 00:46:44 2003 -->
<table><tr><td><a href=/elog/notebooks/silicon/Mon_Jun_16_2003_004638.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-2272  align='top'></td><td>SB2W0L0 dropping readout</a></td></tr></table>
<!-- Author: Rainer -->
<!-- Comment: Mon Jun 16 00;52;57 comment by...Rainer -->
<pre>Try to recover with halt- power cycle - recover run. get trip on switching on on DVDD overcurrent.</pre>


<!-- Comment: Mon Jun 16 01;05;08 comment by...Rainer -->
<pre>.... to no avail ...</pre>