<!-- Title: 11:08:37  Mon Sep 22 2008 -->
<!-- Crewchief: Miguel M (MPC) -->
<!-- Op1: Sasha S (SPL) -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
A new week of data taking


<!-- Date: Mon Sep 22 11:11:53 2008 -->
<h4>Status</h4> 
We are taking data with Silicon. Store 6434 with instantaneous lumi 208 micro b-1*s-1.<br>
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 22 11:16:28 2008 -->
<h3>ISL inhibit; read-back failure</h3> 
There was a read-back problem for ladder IB3W2L0 which Satya fixed with hockerization of Crate 15. Thank you, Satya<br>
<!-- Author: Miguel M -->
<!-- Comment: Tue Sep 23 08;12;15 comment by...Miguel M -->
It seems that there was a real trip; according to the PSGUI 
Mon Sep 22 10:59:15 CDT 2008                        TRIP: ISL  B3  W2 5VDOIM OverCurrent
<br>
<!-- Comment: Tue Sep 23 10;05;20 comment by...Mark -->
The DOIM thing happened as SAR was recovering from the hockerization
<!-- Comment: Sun Sep 28 07;42;08 comment by...MM -->
The voltage for bias-1 was reported as zero, but there was current ~20.
<!-- Date: Mon Sep 22 17:17:29 2008 -->
<h4>SVX/L00 Pinkies; temperature related</h4> 
 
A large number of bias pinkies showed up. It is the known problem of the opening of the resing filter electrovalve and temperature increase.<br>
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 22 20:03:09 2008 -->
<H4>Pinkies; trigger prescale</h4> 
A set of about nine SVX bias pinkies showed up at lumi 108. <br>
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 22 21:29:56 2008 -->
<h4>Imon limits update</h4> 
Limits have been changed for the ladders below.<br><br> 
SVX bias: 
<ul> 
<li>e022</li> 
<li>e191</li> 
<li>e1a1</li> 
<li>e203</li> 
<li>e213</li> 
<li>e251</li> 
<li>e332</li> 
<li>e341</li> 
<li>e501</li> 
</ul> 
 
ISL: 
<ul> 
<li>f112, f503 (bias-1)</li> 
<li>f211  (bias-0) </li> 
</ul>  
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 22 23:42:53 2008 -->
<H4>Jump in bias for SB5W6L1</h4> 
I was paged because of a jump of ~20 appeared in this ladder (<a href= 
'/cgi/elog/elog.pl?nb=2008&action=view&page=798&frame=2&anchor=23:15:43.913'> 
shift e-log</a>). The current plotter shows typical fluctuations of the range 20-30. So, it must be more of the same; it is fine.<br>
<!-- Author: Miguel M -->


<!-- Date: Tue Sep 23 01:49:13 2008 -->
<h3>Anti-trip at EOS for SB2W0</h3> 
At EOS this wedge refused to drop its HV to standby and remain ON (at least as seen in the PSGUI). I was paged and remotely instructed the Ace to try to switch it manually to STBY, after unsucessful attempts, I asked to switch it first to off. Unfortunately the Ace missunderstood and switched off all the SVX detector. So I asked to put SVX back to STBY. The problem was aggrandized with some trips which came afterwards (we was working manually and the coldstart came too late). We hockerize the crate for SB2W0 (crate 1) while keeping SAR disabled and informed MCR that we were ready to drop the beam. They had been holding the beam. After the dump we recovered the trips (hockerizations of Crates 5 & 6 were necessary). Entry in <a href= 
'/cgi/elog/elog.pl?nb=2008&action=view&page=799&frame=2&anchor=00:27:57.271'> 
shift e-log</a> <br>  
<!-- Author: Miguel M -->
<!-- Comment: Tue Sep 23 02;03;40 comment by...Miguel M -->
A similar problem appeared early this year (<a href=
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=389&frame=2&anchor=23:45:11'>
entry</a>). The investigation pointed to SOC as the culprit.



<!-- Date: Tue Sep 23 02:17:19 2008 -->
<h4>TeV quench</h4> 
In shot-setup; the pbars were lost. <br>
<!-- Author: Miguel M -->



<!-- Date: Tue Sep 23 09:33:47 2008 -->
<h3>Investigation for anti-trip incident of SB2W0</h3> 
We have concluded that it was <b>corrupted read-back of the voltages</b>. The plots below show the time evolution of the bias voltage (left) and bias current (right) for one of the ladders (SB2W0L3). The store started about 9:00 and the values look normal, but about 12:00 the voltage jumps to a non-sense extremely high value. It remains at the state until the end of the store. The current looks normal. Similar behaviour is seen in other ladders and the DOIMS (plots not shown).<br> 
<b>The cure is to hockerize the crate.</b><br>
<!-- Author: Ankush / Jen / Miguel M -->
<!-- Comment: Tue Sep 23 09;36;00 comment by...MM -->
Thank you, Ankush and Jen.
<!-- Comment: Tue Sep 23 15;35;45 comment by...Jen -->
Found one other documented occurrence of this on <a href=
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=306&frame=2&anchor=11:07:55'>June 26, 2006</a>.  That was an ISL supply, PS ntuples show its bias channels were reading 2000-3000 volts for ~12 hrs.  They noted the green HV bar in iFix was at 120% instead of 100%, this was the only indication (other than the supply not going to standby at end-of-store).
<!-- Comment: Sun Sep 28 07;59;09 comment by...MM -->
That gives us a clue to catch-up these problems.
<!-- Date: Tue Sep 23 09:35:16 2008 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12902&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12902' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12903&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12903' width='300'></a></center>
<!-- Author: idem -->




<!-- Date: Tue Sep 23 11:07:08 2008 -->
<pre>I tested e372 which is the guy we thought did not return data (according to SVXMon). The channel did give good data while running svxdaq, however it has a cap_id error for chip 89:  
 
 
 0x00000620:  783f7942 7a407b48 7c3e7d3f 7e437f44 c1c10000 00000000 e3726afc 89230017 
0x00000640:  01160216 03140414 05140614 07140814 09140a14 0b130c14 0d130e14 0f141014 
0x00000660:  11141215 13141413 15141614 17141814 19141a14 1b131c14 1d131e14 1f142014 
0x00000680:  21142213 23132413 25132613 27142813 29132a14 2b132c14 2d132e14 2f133014 
0x000006a0:  31143213 33133414 35133614 37133814 39133a14 3b143c14 3d133e14 3f144014 
0x000006c0:  41134214 43144413 45134614 47144814 49144a14 4b144c13 4d144e13 4f145014 
0x000006e0:  51135214 53145414 55145613 57145814 59145a14 5b145c14 5d145e14 5f146014 
0x00000700:  61146213 63136413 65136614 67146814 69146a14 6b146c14 6d146e14 6f137013 
0x00000720:  71147214 73137414 75147614 77147814 79137a13 7b147c13 7d137e13 7f13881c 
0x00000740:  004a013a 023f0339 0434053a 063e0739 083c0934 0a380b38 0c380d3c 0e3c0f39 
0x00000760:  1038113c 1238133b 1434153a 163a173c 183b1936 1a341b3e 1c3a1d3a 1e3e1f39 
0x00000780:  20332139 22402331 2435253c 263e2735 2839293d 2a372b39 2c392d3a 2e3b2f39 
0x000007a0:  303e313c 323f3337 34383539 363b373b 383c393d 3a3a3b34 3c393d3c 3e393f3c 
0x000007c0:  403d4137 423f4333 443c4543 46374742 4838493b 4a3a4b39 4c3b4d3a 4e384f3a 
0x000007e0:  503f513e 523a5338 543a5537 56435738 583a593c 5a3c5b34 5c405d3b 5e395f40 
0x00000800:  603c6139 62386337 643e653e 663a673a 683f693a 6a3a6b35 6c3c6d3b 6e3f6f3c 
0x00000820:  7039713c 7239733a 743e753a 763c773c 783f7938 7a3f7b40 7c377d3a 7e3c7f3e 
0x00000840:  871c0034 0133023c 
 
 according to the PDB this channels has chip 89 sparsed out, ecause it is the first chip and the Error Monitor takes it as reference marking the rest of the chips as bad. I guess that is why it appears in the Bit Reporter although it is masked. 
</pre>
<!-- Author: sergo -->


<!-- Date: Tue Sep 23 11:15:02 2008 -->
<pre>e564 is dead - not returning any data. This ladder had bits 6,7 low in the past, thus returning garbage data, it was declared "unfixable" and masked in SVXMon. This is why we did not notice that it stopped returning any data. If there is desire to try to revive it we can do a fiber swap during next acccess. However past RX swaps were unsuccessful. </pre>
<!-- Author: sergo -->


<!-- Date: Tue Sep 23 11:28:00 2008 -->
<pre>e493 returned data in svxdaq however bit 1 high error has re-appeared, which is causing the weird (step like) adc distributions. </pre>
<!-- Author: sergo -->


<!-- Date: Tue Sep 23 12:40:28 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12904&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12904  width='300' align='top'></a></td><td><h3>Hockerizations per Week</h3> 
 
Just an updated plot showing the number of hockerizations per week. As with the previous plot, I was not able to remove spurious entries (mostly due to testing the AutoHocker stuff I think), so some weeks seem to have a lot more hockerizations than they actually do.</td></tr></table>
<!-- Author: Mark -->
<!-- Comment: Tue Sep 23 12;54;07 comment by...Mark -->
The spike in the recent week is due to a couple of access we had. It seems that even when we set it to manual hocker mode, iFix still tries to hockerize the crate, which of course produces an entry in the log file (even though no hockerization was done). Perhaps there is a way to disable iFix from sending the request?


<!-- Date: Tue Sep 23 13:10:49 2008 -->
<h2>New Power Supply Loadbox has Arrived!</h2> 
 
We got the first loadbox from the FCC guys yesterday, to start testing and playing around with. It is not quite finished (needs labels and warning stickers and such), and there is still much firmware to develop (yes, it has a processor!), but we are able to test it out, put it through it's paces, and try to break it as best we can :)
<!-- Author: Mark -->



<!-- Date: Tue Sep 23 20:23:47 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12905&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12905  width='300' align='top'></a></td><td><h3>Corrupted read-back in shot-setup</h3> 
Bias-0 current for ladder IB4W5L4 jumped to a crazy value during shot-setup. I hockerized Crate 12.</td></tr></table>
<!-- Author: Miguel M -->


<!-- Date: Tue Sep 23 20:31:16 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12906&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12906  width='300' align='top'></a></td><td><h3>Store 6442</h3> 
Started ~21:20 with initial instantaneous lumi ~270 micro b-1*s-1. Losses below 2 KHz. Silicon is in. We are taking data. SpyMon plot and SVXMon plots look as usual.</td></tr></table>
<!-- Author: Miguel M -->


<!-- Date: Tue Sep 23 20:37:29 2008 -->
Bias IMon limit for SB0W8L2 has been updated.<br>
<!-- Author: Miguel M -->


<!-- Date: Wed Sep 24 11:49:29 2008 -->
<h4>Test of trigger table</h4> 
Tom W and Laura are testing [10,1139,728]. We first run non_silicon. Now they are runing with silicon; the L1 trigger rates are around 19 KHz. They will keep this table for the rest of the store (estimated EOS 13-14:00). <br>
<!-- Author: Miguel M -->
<!-- Comment: Wed Sep 24 11;51;11 comment by...MM -->
In fact, the new table is now the default (<a href=
'/cgi/elog/elog.pl?nb=2008&action=view&page=803&frame=2&anchor=11:38:46.536'>
shift e-log</a>).


<!-- Date: Wed Sep 24 11:59:44 2008 -->
IMon bias upper limit  has been updated for SB2W8L2<br>
<!-- Author: Miguel M -->
<!-- Comment: Thu Sep 25 02;54;36 comment by...MM -->
Also for SB5W6L1.
<!-- Comment: Thu Sep 25 08;14;59 comment by...MM -->
Also for SB4W2L1
<!-- Comment: Thu Sep 25 17;46;37 comment by...MM -->
And BIAS-0 limit for IB4W1L0.
<!-- Comment: Thu Sep 25 18;00;08 comment by...MM -->
And BIAS-0 limit for IB5W2L1.
<!-- Date: Wed Sep 24 14:37:39 2008 -->
ISL IB5W5L1 tripped on Bias0 overvoltage during ramping down at EOS. I ahve waited until beams have been dropped and enabled SAR. The ladder continued tripping on ovv. I hockerized CAEN crate 16 and this fixed the faulty PS. I have also noticed that PSgui was very slow so I restarted it.
<!-- Author: SashaS -->


<!-- Date: Wed Sep 24 14:49:26 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12907&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12907  width='300' align='top'></a></td><td>Our e-mail warnings about ISL CCE3's PCOT have been getting more frequent (or at least it seems that way), and the second alarm one came in yesterday.  I reduced the ISL CCE3 control valve from 59% first to 55% (no change, valve is sticky), and then down to 53% (big step, valve unsticks).  These plots show the CCE3 PCOT during this change on the real-time plot, and the historical flow rates for the last 90 days.  CCE3 used to have a flow of 2.16 lpm before the vacuum incident, which went down to 1.96 lpm after flows were recovered.  We got pinkies at that flow rate, and opened the valves and got 2.3 lpm.  The pinkies subsided, but we were up against the PCOT warnings.  Over time, the flow increased on its own to 2.5 lpm, but the PCOT remained erratic.  Now that we have more headroom with the flows, we decided to use a bit of it to keep the pressure farther from the PCOT warnings.</td></tr></table>
<!-- Author: Tom Junk -->
<!-- Comment: Wed Sep 24 16;36;16 comment by...Miguel M -->
That was a wonderful idea. The CCE3 pressure at COT face is now 11.8 psi and looks safe below the warning limit of 13.3 psi. The flow has stepped down to 2.2 lpm, very close to the value before the vacuum pump incident (2.16 lpm).
<!-- Date: Wed Sep 24 16:02:24 2008 -->
<h3>TeV quench 2</h3> 
Just after loading all pbars and initiating collisions.<br>
<!-- Author: Miguel M -->




<!-- Date: Thu Sep 25 03:09:58 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12908&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12908  width='300' align='top'></a></td><td><h3>Store 6443; low lumi</h3> 
Started ~2:45 am with low initial instantaneous lumi ~198 micro b-1*s-1. They started this store with only 200E+10 pbars in recycler; perhaps as result of the two quenches this week in shot-setup. Losses are very low ~ 1 KHz. SpyMon plot looks as usual.</td></tr></table>
<!-- Author: Miguel M -->
<!-- Comment: Thu Sep 25 03;16;38 comment by...MM -->
SVXMon plots are now fine


<!-- Comment: Thu Sep 25 08;21;03 comment by...MM -->
From todays Run Coordinator e-log "It was determined that bright Pbars caused the quench".
<!-- Date: Thu Sep 25 03:22:26 2008 -->
<h4>Auto-hockerization; Crate 7</h4> 
L00/ISL; successful auto-recovery.<br>
<!-- Author: MM -->




<!-- Date: Thu Sep 25 10:35:10 2008 -->
<h3>RF station trip</h3> 
The SciCo told me that the station was going to be off. I checked TeVMon and the last alarm to go away was "TAGI Gap Intensity 24-25". Then we cleared the Beam Permit and put Silicon on. The losses are below 0.5 KHz. <br>
<!-- Author: Miguel M -->
<!-- Comment: Sun Sep 28 11;41;44 comment by...MM -->
A plot of the incident is posted <a href=
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=11:40:05'>
below</a>.
<!-- Date: Thu Sep 25 15:35:22 2008 -->
<pre>Its not a surprize that f544 does not have ADC plot (see monitoring report from last week) - it is off and out of the HW DB.</pre>
<!-- Author: sergo -->
<!-- Comment: Thu Sep 25 15;36;49 comment by...sergo -->
<pre>so is f043. </pre>

<!-- Comment: Thu Sep 25 15;37;18 comment by...sergo -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>Please check status of the ladder before reporting it bad.</pre></td></tr></td></table>

<!-- Date: Thu Sep 25 15:59:31 2008 -->
Entered in Silicon database: <ul>
<li>Ladder: IB5W3L3 
<li>problem n: 2573 
<li>Status: needs_work
<li>Type: Other
<li>Action: Diagnosis_update
<li>entered by: sergo 
<li>description is: Bit2 low problems reappeared. Please disregard earlier entry ('fixed after RX swap'), as far as I remember we never swapped RX for this ladder, probably the entry was made by mistake.  Will do fiber swap during the shutdown next week.  
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=ISL&wedge=B5W3&layer=L3">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Thu Sep 25 16:40:51 2008 -->
<pre> Reason for e250 bit errors: this ladder has no light at bore for bit 6. Low bit 6 causes 7f-> 3f, and since there is no 7f (last channel ID) for last chip (a0), it drops the readout with c2c2.  Suggest keeping this ladder on (the data is partially good) however there is nothing we can do to fix bit errors.</pre>
<!-- Author: sergo -->




<!-- Date: Thu Sep 25 18:11:24 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12909&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12909  width='300' align='top'></a></td><td><h3>Store 6448</h3> 
Started ~18:00 with initial instantaneous lumi ~272 micro b-1*s-1. Losses below 4 KHz. SpyMon plot looks as usual.</td></tr></table>
<!-- Author: Miguel M -->
<!-- Comment: Thu Sep 25 18;16;26 comment by...MM -->
SVXMon plots look now fine.


<!-- Date: Thu Sep 25 22:36:06 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12911&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12911  width='300' align='top'></a></td><td><h3>Electron Lens Trip</h3> 
It was off for about three minutes. The Ace noticed that TeVMon was in warning (<a href= 
'/cgi/elog/elog.pl?nb=2008&action=view&page=807&frame=2&anchor=21:00:12.880'> 
shift e-log</a>) and the crew called MCR; then they did the recovery (<a href=http://www-bd.fnal.gov/cgi-mcr/elog.pl?nb=2008&action=view&page=731&anchor=210136&hilite=21:01:36- target=_top>MCR e-log</a>). If the trip had persisted, TeVMon would have gone to alarm state after ten minutes.<br> 
(Plot stolen from Ace) 
</td></tr></table>
<!-- Author: Miguel M -->


<!-- Date: Fri Sep 26 02:43:17 2008 -->
<h4>No data from SB3W7L2</h4> 
This ladder was yellow in SVXMon; HRR did not cure it. I instructed the crew to restart SVXMon. The ADC distributions are empty. The first chip is sparsed out. Since the DAQ has been very quiet in this owl shift, I did not dare to try restarting a new run just for one ladder. I made an entry in PDB as "needs_work" to hide the problem for the time being. Let us see if the problem disappears in the following runs. This ladder has a long history of bias and AVDD2 issues.<br>
<!-- Author: Miguel M -->
<!-- Comment: Fri Sep 26 03;00;56 comment by...MM -->
The DQMon stopped complaining too (SVX was red).


<!-- Comment: Fri Sep 26 16;44;53 comment by...MM -->
This ladder was in Sergos talk of this afternoon.
<!-- Comment: Sat Sep 27 11;59;00 comment by...MM -->
It was studied in SVXDAQ this week, <a href=
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=11:07:08'>
entry</a> above.

<!-- Comment: Sat Sep 27 12;11;09 comment by...Miguel M -->
I updated the PDB thread as "needs_observation". I hope that it remains blue in SVXMon.
<!-- Date: Fri Sep 26 10:27:51 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12912&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12912  width='300' align='top'></a></td><td>ISL DI resin bed seems to have lost its ability to 
deionize; the conductivity hasn't yet started going up, 
but it isn't going down and the ISL DI valve is open.</td></tr></table>
<!-- Author: Tom Junk -->



<!-- Date: Fri Sep 26 18:30:57 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12913&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12913  width='300' align='top'></a></td><td><h3>Store 6452</h3> 
Started ~18:15 with initial instantaneous lumi ~280 micro b-1*s-1. Losses below 2 KHz. SpyMon plot looks as usual.</td></tr></table>
<!-- Author: Miguel M -->
<!-- Comment: Fri Sep 26 18;33;08 comment by...MM -->
SVXMon plots look now fine.


<!-- Date: Sat Sep 27 07:06:24 2008 -->
<h3>TeV Quench 3</h3> 
The trilogy of the week. All pbars lost in shot-setup.<br>
<!-- Author: Miguel M -->


<!-- Date: Sat Sep 27 08:25:08 2008 -->
<h4>Zero DVDD current for SB5W9L1; hockerized</h4> 
The zero current was reported by IMon and PSGUI. Coldstarts could not fix it. It could be corrupted read-back. I hockerized Crate 13 and now the current looks up.<br>
<!-- Author: Miguel M -->


<!-- Date: Sat Sep 27 08:46:35 2008 -->
<h4>Alarms wont mute</h4>
I silenced SVX IFIX alarms 2-4, but even so the audible alarm went off after the hockerization. This is not the first time that I experience this. This should be investigated.<br>
<!-- Author: Miguel M -->


<!-- Date: Sat Sep 27 11:55:31 2008 -->
<h3>Pinky work</h3> 
Two bursts of pinkies occurred in the owl shift; one for temperature change (resin filter e-valve), the other for dynamic trigger prescale change.  
IMon upper limits have been updated for the following ladders: 
<ul> 
<li>SVX:<br> 
e093, e024, e062;<br> 
e112, e154, e121;<br> 
e363, e383, e300, e393, e3b0;<br> 
e494, e491, e412, e401, e4b0;<br> 
e580 
</li> 
<br> 
<li> L00  (BIAS-0):<br> 
f832, f842, f812;<br> 
fa11, fa41, fa02, fa22 
</li> 
<br> 
<li>ISL (BIAS-0):<br> 
f131 
</ul>  
<!-- Author: Miguel M -->




<!-- Date: Sat Sep 27 19:30:32 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12914&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12914  width='300' align='top'></a></td><td><h3>Store 6457</h3> 
Started ~19:05 with initial instantaneous lumi ~245 micro b-1*s-1. Very low losses ~ 1 Khz. SpyMOn plot looks as usual.</td></tr></table>
<!-- Author: Miguel M -->


<!-- Date: Sat Sep 27 21:47:13 2008 -->
<h4>Auto-hockerization Crate 2</h4> 
An auto-hockerization occurred and auto-recovering in progress, but SAR was complaining that could not send config to a FIB. I noticed that the FIB was in error in Run Control and naively thought that it had a problem and tried to fix it by shepherding the crate while I disabled SAR. Those were mistakes; as I understood later when I cooled down, the only problem was that RC was not in Halt, but in Active state (red color). Since the FIB remained in error and I was still confused, we ended the run. Then I completed the recovery with SAR and put Silicon back in DAQ.<br> 
Once I learned the lesson, I instructed the Ace to put RC in halt during auto-hockerizations or trips.<br>
<!-- Author: Miguel M -->


<!-- Date: Sun Sep 28 07:02:59 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12915&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12915  width='300' align='top'></a></td><td><h3>Store 3457</h3> 
Started ~6:45 am with initial instantaneous lumi ~ 242 micro b-1*s-1. Losses below 1 KHz. SpyMon and SVXMon plots look as usual.</td></tr></table>
<!-- Author: Miguel M -->
<!-- Comment: Sun Sep 28 12;52;49 comment by...MM -->
Typo: Store 6458
<!-- Date: Sun Sep 28 07:30:47 2008 -->
<h3>Auto-hockerization Crate 15</h3> 
This ISL/L00 crate was recovered by SAR.<br>
<!-- Author: Miguel M -->




<!-- Date: Sun Sep 28 11:40:05 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12916&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12916  width='300' align='top'></a></td><td><h3>RF trip of last Thursday</h3> 
This plots shows how the RFSUM (red line) drops at the time when the HV of RF Station No. 6 for protons trips off(red circles). RFSUMA increases (yellow line). The abort gap losses go up and down and stabilize. The HV of Station No. 5 (yellow circles) is shown as reference. The axis of the right side gives the scale for the HV of the stations.</td></tr></table>
<!-- Author: Miguel M -->




<!-- Date: Sun Sep 28 14:47:40 2008 -->
<h3> ADC Monitoring Summary (Run267275-267371) </h3>  
I have checked ADCMon plots during this week from 267275 to 267371. The problems common to those runs are: <BR> 
  
 
<ul>  
<li> Ladders e052 and e372, no reference plot.   
<li> f302 weird distribution. See previous Aug. 25th silicon <a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=419&frame=2&anchor=08:14:41'>entry </a>. 
  
<li> f401,f403,f404 no data. 
<li> f451,f453 no data. 
<li> f533,f544 no data.  
</ul> 
 
  
<!-- Author: Roberto -->


<!-- Date: Sun Sep 28 20:05:25 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12917&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12917  width='300' align='top'></a></td><td><h3>Huge number of trips in SVX</h3> 
At ~16:00 about 200 ladders tripped in SVX (60 % of the detector). Most of them are reported as DVDD overcurrent. SAR was able to recover a great deal of it, but not all. I was paged and came to Control Room and had to hockerized seven crates (Crates 1,2,6,9,10,13,14). I did it gradually, one by one, and waiting for SAR to recover the HV before hockerizing the next. I instructed the Ace to run Non_Silicon.<br> 
I still do not understand the cause of the incident. The Logic Analizer was still running after the incident and the SRC crate had no error, so and SRC glitch is ruled out.<br> 
The screen shot corresponds to a moment before the combined manual + automatic recovery (a fraccion of the trips). IMon was stuck when I arrived at Control Room.</td></tr></table>
<!-- Author: Miguel M -->


<!-- Date: Sun Sep 28 20:25:22 2008 -->
<pre>From the PSGUI logfile about the incident: 
 
 
Sun Sep 28 15:58:44 CDT 2008     TRIP: SVX  B3  W6 DVDD3 OverCurrent 
Sun Sep 28 15:58:44 CDT 2008     TRIP: SVX  B3  W6 DVDD4 OverCurrent 
Sun Sep 28 15:58:44 CDT 2008     TRIP: SVX  B4  W4 DVDD2 OverCurrent 
Sun Sep 28 15:58:44 CDT 2008     TRIP: SVX  B4  W4 DVDD4 OverCurrent 
Sun Sep 28 15:58:44 CDT 2008     TRIP: SVX  B4  W5 DVDD4 OverCurrent 
Sun Sep 28 15:58:44 CDT 2008     TRIP: SVX  B4  W6 DVDD1 OverCurrent 
Sun Sep 28 15:58:44 CDT 2008     TRIP: SVX  B5  W7 AVDD0 OverCurrent 
Sun Sep 28 15:58:56 CDT 2008     TRIP: SVX  B0  W0 DVDD0 OverCurrent 
... 
Sun Sep 28 15:58:56 CDT 2008     TRIP: SVX  B5  W11 DVDD2 OverCurrent 
Sun Sep 28 15:58:56 CDT 2008     TRIP: SVX  B5  W11 DVDD3 OverCurrent 
Sun Sep 28 15:58:56 CDT 2008     TRIP: SVX  B5  W11 DVDD4 OverCurrent 
...(Totalizing 201 overcurrents) ... 
 
Sun Sep 28 16:00:00 CDT 2008     Software protection trip: High DVDD Current  SVX B4 W4 L3 current 445.0 softLimit 275.0 
Sun Sep 28 16:00:04 CDT 2008     Software protection trip: High DVDD Current  SVX B4 W4 L4 current 950.0 softLimit 350.0 
Sun Sep 28 16:00:08 CDT 2008     Software protection trip: High DVDD Current  SVX B4 W5 L1 current 399.0 softLimit 150.0 
Sun Sep 28 16:00:13 CDT 2008     Software protection trip: High DVDD Current  SVX B4 W5 L3 current 575.0 softLimit 275.0 
... 
Sun Sep 28 16:05:25 CDT 2008     Software protection trip: High DVDD Current  SVX B5 W6 L4 current 410.0 softLimit 350.0 
Sun Sep 28 16:06:28 CDT 2008     Software protection trip: High DVDD Current  SVX B3 W5 L4 current 455.0 softLimit 350.0 
Sun Sep 28 16:11:42 CDT 2008     Software protection trip: High DVDD Current  SVX B3 W5 L4 current 670.0 softLimit 350.0 
Sun Sep 28 16:16:01 CDT 2008     Software protection trip: High DVDD Current  SVX B3 W5 L4 current 690.0 softLimit 350.0 
...(Totalizing 51 software protection trips) 
 
 
 
</pre>
<!-- Author: Miguel M -->


<!-- Date: Sun Sep 28 21:00:31 2008 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12918&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12918  width='300' align='top'></a></td><td><h3>Store 6459</h3> 
Started ~20:20 with initial instantaneous lumi ~242 micro b-1/s. Losses below 2 KHz. SpyMon and SVXMon plots look as usual.</td></tr></table>
<!-- Author: Miguel M -->


<!-- Date: Sun Sep 28 21:26:41 2008 -->
<h3>Low DOIM currents for SB0W7</h3> 
I noticed after the general trip that these currents are about 7% below their normal ranges:<br> 
<br> 
DOIM-2V:  Current 895  (limits 1065 and 1160)<br> 
DOIM-5V:  Current 1745 (limits 1875 and 1955)<br> 
<br> 
I hockerized Crate 6 between stores, but that did not fix the problem. I have marked the DOIMS as ignore in IMon.<br>
<!-- Author: Miguel M -->
<!-- Comment: Sun Sep 28 21;49;54 comment by...MM -->
The voltages look normal, 2.01 and 5.20 V.


<!-- Date: Sun Sep 28 22:20:55 2008 -->
<h3>Corrupted read-back; zero voltage</h3> 
Ladders in wedge IB5W2 were reported with zero voltage for DVDD. Also all bias currents dropped. I was in Control Room and hockerized Crate 12.<br>
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 29 09:04:06 2008 -->
IMon upper limit updated for IB3W1L2 (bias-0).
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 29 09:20:49 2008 -->
<h4>IMon logfiles?</h4> 
Where is IMon supposed to write its logfiles?<br> 
The "Options" button points to a directory named "logfiles/" (does not precise the full path).<br> 
I found in the TWiki the path:<br> 
/cdf/onln/data/svxii/data1/datalog/ps/imon/logfiles<br> 
and the <a href= 
'https://www-cdfonline.fnal.gov/internal/ops/svx/data/data1/datalog/ps/imon/logfiles/Sep29.2008.log'> 
webpage</a> of last file (todays).<br> 
But it happens that todays files are surprisingly empty (which must be wrong) and LastUpdate.log stopped about midnight.<br><br> 
Also, making a search in the e-log, I found the path:<br> 
~svxii/imon_dev/logfiles/<br> 
which has the right file for today (which the information I expected) and the right LastUpdate.log<br> 
  
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 29 09:23:26 2008 -->
 
<h2>Summary of the Week</h2> 
 
<h3>General comments:</h3> 
<ul> 
<li> Smooth running</li> 
<li>Corrupted read-back issues</li> 
<li> 3 quenches during shot-setup (pbars lost)</li> 
<li>SVX big trip</li> 
</ul> 
 
<h3>Monday</h3> 
<ul> 
<li> Inherited Store 6434 </li> 
<li> Corrupted read-back: ladder with zero bias VOLTAGE, inhibit; hockerized  (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=11:16:28'> 
entry</a>)</li> 
<li> Burst of pinkies; SVX resin filter e-valve </li> 
<li> Burst of pinkies: trigger prescale at 110 microb-1/s</li> 
</ul> 
 
<h3>Tuesday</h3> 
<ul> 
<li> Corrupted read-back at EOS, wedge "anti-trip"; hockerization (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=09:33:47'> 
entry</a>) </li> 
<li> TeV Quench 1</li> 
<li> Corrupted read-back: ladder with crazy bias CURRENT; hockerized (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=20:23:47'> 
plot</a>) </li> 
<li> Store 6442 </li> 
</ul> 
 
<h3>Wednesday</h3> 
<ul> 
<li>Ladder trip at EOS, overvoltage; hockerized (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=14:37:39'> 
entry</a>) </li> 
<li> TeV Quench 2 </li> 
</ul> 
 
 
<h3>Thursday</h3> 
<ul> 
<li> Store 6443 </li> 
<li> Auto-hockerization; Crate 7 (L00/ISL) </li> 
<li> RF station trip (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=11:40:05'> 
plot</a>) <li> 
<li> Store 6448</li> 
<li>Electron lens trip (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=22:36:06'> 
plot </a>)</li> 
</ul> 
 
 
<h3>Friday</h3> 
<ul> 
<li>No issues (phew!)</li> 
<li>Silicon Meeting</li> 
<li>Store 6452 </li> 
</ul> 
 
 
<h3>Saturday</h3> 
<ul> 
<li> Burst of pinkies; SVX resin filter e-valve </li> 
<li> Burst of pinkies: trigger prescale at 110  microb-1/s</li> 
<li>TeV Quench 3 (Trilogy completed)</li> 
<li>Corrupted read-back: zero DVDD current; hockerization (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=08:25:08'> 
entry</a>) 
 </li> 
<li>Store 6457</li> 
<li>Auto-hockerization, Crate 2 (SVX/ISL); my blunder (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=21:47:13'> 
entry</a>) </li> 
</ul> 
 
 
<h3>Sunday</h3> 
<ul> 
<li>Store 6458</li> 
<li>Auto-hockerization, Crate 15 (ISL/L00)</li> 
<li>SVX trip (60 % of detector) (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=20:05:25'> 
entry</a>)  </li> 
<li>Low DOIM currents for a wedge (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=21:26:41'> 
entry</a>)</li> 
<li>Store 6459</li> 
<li>Corrupted read-back for a wedge (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=22:20:55'> 
entry</a>) </li> 
</ul> 
 
 
<h3>Miscelaneous</h3> 
<ul> 
<li>Audible SVX Alarms wont mute (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=08:46:35'> 
entry</a>) </li> 
 
<li> Request for resin exchange for ISL(<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=10:27:51'> 
plot</a>) </li> 
 
<li> ISL Pressure at COT face (East) relieve (last Fridays Meeting) </li> 
 
<li><a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=14:47:40'> 
Monitoring</a>  </li> 
 
 
<li> Quiet time studies with SVXDAQ  (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=11:07:08'> 
entry</a>, <a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=15:35:22'> 
more</a>) (last Fridays Meeting) </li> 
 
<li> Hockerizations per week (<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=424&frame=2&anchor=12:40:28'> 
entry</a>) </li>  
 
<li> New toy: PS loadbox (last Fridays Meeting) </li> 
 
 
 
</ul> 
  
<!-- Author: Miguel M -->


<!-- Date: Mon Sep 29 09:30:50 2008 -->
The run which was interrupted yesterday by the SVX trip was marked incorrectly BAD by the crew. I noticed it after reading the information of the store. So, I complained in the  
<a href= 
'/cgi/elog/elog.pl?nb=2008&action=view&page=817&frame=2&anchor=01:09:03.955'> 
shift e-log </a>. After todays 8:00 am meeting, the OpsManager talked to me and I explained.<br>  
<!-- Author: Miguel M -->