<!-- Title: 11:15:18  Mon Aug  1 2005 -->
<!-- Crewchief: Jen -->
<!-- Op1: Sebastian (SPL) -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
New week, new month...


<!-- Date: Mon Aug 1 11:17:04 2005 -->
<center><a href='/elog/notebooks/silicon/Mon_Aug__1_2005_111549.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7135' ></a><a href='/elog/notebooks/silicon/Mon_Aug__1_2005_111648.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7137' ></a></center>
<!-- Author: Jen & Sebastian -->
<!-- Comment: Mon Aug  1 11;21;48 comment by...Jen & Sebastian -->
<br><b>f223</b>
<br>
Tried running f223 with SVXDaq.  The left plot shows f223 with the default ISL integrator bit settings, 1010.  The right plot shows with the FE oscillation settings, 1111, for chips 0-3.<br>


<!-- Date: Mon Aug 1 12:02:28 2005 -->
<table><tr><td><a href=/elog/notebooks/silicon/Mon_Aug__1_2005_120225.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7139  align='top'></a></td><td>More on f223...</td></tr></table>
<!-- Author: Jen & Rong -->
<!-- Comment: Mon Aug  1 12;07;20 comment by...Jen & Rong -->
So we drafted Rong for his expertise to take another look at f223, since we didn't see any difference from changing the ISL integrator bits.  The above plot shows f223 with the ISL bits set to 1111 for chips 0-3 AND chips 12-15.  Again, it didn't make a difference in the noise for chips 0-3, so that may be a lost bias problem rather than FE oscillation.  But it does seem to reduce the noise on chips 12-15.<br><br>
So for the next store, ISL bits have been set to 1111 for chips 0-3 and 12-15 on f223.  We'll see if that makes any difference in the noise levels.  As a consequence of the changes, f223 AVDD current has gone up from about 1120 to about 1205.  It's currently set to IGNORE in IMON, we'll change the limits after the store starts.<br>


<!-- Date: Mon Aug 1 12:16:55 2005 -->
After finishing with f223, did a coldstart of the system.  Also restarted PSGui, just to be safe.
<!-- Author: Jen -->


<!-- Date: Mon Aug 1 12:59:43 2005 -->
<h3>EVB test</h3> 
 
<p> 
This morning Markus and I did some more testing on the new EVB.  
We still saw errors coming from VRB_43 on b0svx05, even though  
we swapped the VRB (in b0svx01 we swapped the wrong VRB...). 
We talked to SCN and he pointed that the two VRBs giving problems 
have missing wedges: e17 and e54. Furthermore, e17 was giving problems more often than e54, and e17 has only 4 ladders of the working hdi, while e54 has 5. Markus is looking into this. </p>  
 
<p> 
The second kind of probelms show up after some 10k 's events are 
taken. We get BTO but were not able to trace who is pulling busy. 
Again SCN gave some ideas which we are going next (EOS). 
</p>
<!-- Author: Sebastian -->


<!-- Date: Mon Aug 1 13:52:44 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7142&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7143  width='300' align='top'></a></td><td>SVT spymon for store 4313.  Initial Lum = 111, losses are low.  SVXMon looks good too.  Started out with high lum table, L1A is only ~6 kHz.</td></tr></table>
<!-- Author: Jen -->


<!-- Date: Mon Aug 1 14:19:28 2005 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7144&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7144' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7145&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7145' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7146&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7146' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7147&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7147' width='300'></a></center>
<!-- Author: Jen -->
<!-- Comment: Mon Aug  1 14;21;12 comment by...Jen -->
<h2>Strip plots for f223</h2>
Strip charge and RMS plots for f223, chips 0-3 (1st and 3rd plots) and chips 12-15 (2nd and 4th plots), after changing their ISL bits to 1111.<br>


<!-- Comment: Mon Aug  1 15;09;56 comment by...Rong-Shyang -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7150&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7150' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7151&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7151' width='300'></a></center> <BR>
Together with mean charge plots, f223 is still having problem. I tend to believe it's just bias problem.

<!-- Date: Mon Aug 1 14:24:35 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7148&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7149  width='300' align='top'></a></td><td>History plot for IB2W2L3 AVDD current after making changes to ISL bits.  Current looks stable, will change limits.</td></tr></table>
<!-- Author: Jen -->

<!-- Date: Mon Aug 1 14:28:01 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: IB2W2L3 
<li>problem n: 1499 
<li>Status: needs_observation
<li>Type: FE_amp_osc
<li>Action: IMON_limits
<li>entered by: jpursley 
<li>description is: Modify limits on AVDD after changing ISL bits as noted in previous entry.<table border=1>
<tr><th>IMON LIMITS</th><th>Old</th><th>New</th></tr>
<tr><td><b>Nominal</b></td><td>1139.65</td><td>1205</td></tr>
<tr><td><b>Lower</b></td><td>1090</td><td>1155</td></tr>
<tr><td><b>Upper</b></td><td>1140</td><td>1210</td></tr></table> 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=ISL&wedge=B2W2&layer=L3">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Mon Aug 1 14:30:38 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: IB2W2L3 
<li>problem n: 1499 
<li>Status: needs_observation
<li>Type: FE_amp_osc
<li>Action: IMON_limits
<li>entered by: jpursley 
<li>description is: Oops, used wrong nominal.  Again, changing AVDD limits after changing ISL bits.<table border=1>
<tr><th>IMON LIMITS</th><th>Old</th><th>New</th></tr>
<tr><td><b>Nominal</b></td><td>1205</td><td>1235</td></tr>
<tr><td><b>Lower</b></td><td>1155</td><td>1185</td></tr>
<tr><td><b>Upper</b></td><td>1210</td><td>1240</td></tr></table> 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=ISL&wedge=B2W2&layer=L3">report</a></ul>
<!-- Author: Problem DB -->




<!-- Date: Mon Aug 1 16:01:02 2005 -->
<h3>Problems with VRB_32 in b0svx03</h3> 
 
<p> 
Two ladders were yellow in SVXMon, e360 and e361. We first 
looked at the glinks but they were synch'ed. Then we rebooted the FIB, but still the data looked like: 
<pre> 
     3536: a1290903 7f06a029 e360c1c1 d0d006fe  
     3540: d0d0d0d0 d0d0d0d0 d0d0d0d0 00f38329  
     3544: 360e3503 38f0373a 3a2939d1 3c043b0f  
     3548: 8229d0d0 a129d0d0 a029d0d0 230c2207  
     3552: 25f32408 270a260a 2a092806 2c1c2b0d  
     3556: 2e082d0c 30092f0c 320c310a 340b330a  
     3560: 36073512 3809370c 3a0f3910 3c0c3b0e  
     3564: 3e083d0c 400d3f11 4206410d c1c17f02  
     3568: c4c4c4c4 c4c4c4c4 c4c4c4c4 c4c4c4c4  
     3572: c4c4c4c4 c4c4c4c4 c4c4c4c4 c4c4c4c4  
</pre> 
But there was no new hole in SVT. So we rebooted b0svx03, 
and after cleaning the EVB we are running fine. SVT 
looks fine too.<br>
<!-- Author: Jen/Sebastian/Chris/Rong -->
<!-- Comment: Mon Aug  1 16;01;55 comment by...Jen/... -->
WE rebooted b0svx03 because the problem was with the VRB.

<!-- Comment: Mon Aug  1 17;03;49 comment by...SCN -->
This VRB was somehow put into WYSIWYG mode (What You See Is What You Get) where it does not strip away 0xd0d0 between the fib header and the data and also does not remove trailing c4c4 after the c1c1.
(This is a little confusing because the data as presented above is short-swapped: if not you would see 
<pre>c1c1e360 06fed0d0 
d0d0d0d0 d0d0d0d0 d0d0d0d0 832900f3
... ... ... ...   7f02c1c1 c4c4c4c4 ... </pre>)   
It is a little odd since during CONFIG the <ahref="https://www-cdfonline.fnal.gov/cgi-bin/cvsweb.cgi/~checkout~/svxii/si_ss/server/src/svx_proc.c?rev=1.84&hideattic=1&sortbydate=0"> code</a>
does <pre>
  /* Some day we should put these into the state, *
   * but for now, enable the L1A timeouts, disable the WYSIWYG */
  for (i=0; i<10; i++) {
    vrb_enable(VRB_L1_TO_BASE + i, vrbM->slot, &error);
    vrb_disable(VRB_WYSIWYG_BASE+i, vrbM->slot, &error);
  }
</pre>
i.e. explicitly disables WYSIWYG.  This has happened before, but not as frequently as eg. a solar eclipse or a party a Rainer's house.


<!-- Comment: Tue Aug  2 09;50;18 comment by...SCN -->
Oops: <a href="https://www-cdfonline.fnal.gov/cgi-bin/cvsweb.cgi/~checkout~/svxii/si_ss/server/src/svx_proc.c?rev=1.84&hideattic=1&sortbydate=0"> code</a>
<!-- Date: Mon Aug 1 17:31:07 2005 -->
<h2>Electron lens tripped off</h2> 
... see plots in <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&action=view&page=639&frame=2&anchor=17:14:19.232'>main elog</a>.  Shift crew immediately put silicon to standby and paged me.  MCR called shortly afterwards to say they had experts coming in to look at it and advised the losses were low enough that we could keep silicon on.  I chose to keep silicon at standby until the lens was back on.  Luckily, they got the lens back on almost immediately.  Abort gap losses spiked, but not badly enough to set off warnings in TevMon.  After losses stabilized, I gave permission to ramp silicon.<br> 
MCR hasn't yet given a reason for the lens tripping off.<br>
<!-- Author: Jen -->


<!-- Date: Mon Aug 1 17:37:27 2005 -->
<h2>Problems with b0svx08:</h2> 
After bringing silicon back up, we tried to start a new run, but b0svx08 failed config.  There was no message in the errorlogger about it; the yellow pop-up window just said that b0svx08 had failed config after trying for 120 seconds.  The ace had aborted the run before calling me over, svx08 was stuck and wouldn't abort either.<br> 
I pinged b0svx08, it responded perfectly.  So then I opened a minicom window and looked at the processes; Messenger was there, status PEND+T.  I decided to reboot svx08.  Then we cleaned up EVB and started a new run successfully.<br>
<!-- Author: Jen -->



<!-- Date: Tue Aug 2 10:00:20 2005 -->
<pre>ADCMon had no data for two entire wedges in runs 202050 and 202052: 
 
e160 e161 e162 e163 (e164 already bad) 
e550 e551 e552 e553 e554  
 
ADCMon sometimes seems to have trouble parsing corrupted data and freaks out, so I checked bitreporter.  The two runs in question are grayed, which usually indicates a ladder was unintegrated during those runs, but it also claimed to see "no data" at a 2% rate, which would only happen if the ladder was integrated. 
 
In a subsequent run, the data in ADCMon is back (you can also check the SVT plot for store 4313 above.) 
 
How are B1W6 and B5W5 related?</pre>
<!-- Author: boveia -->
<!-- Comment: Tue Aug  2 10;19;37 comment by...Sebastian -->
They are not related, but during the EVB test we took two VRBs out,  so the these two wedges were out. <br>
We took out VRB_43 (svx05) e54, e55; and VRB_20 (svx01) e16, e17.
(e17 and e54 are dead wedges). See <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=259&frame=2&anchor=02:30:08'>
here</a>.

<!-- Comment: Tue Aug  2 10;26;01 comment by...s -->
Also <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=260&frame=2&anchor=12:59:43'>
here</a>.


<!-- Comment: Tue Aug  2 11;23;32 comment by...boveia -->
<pre>Ok, good to know the monitoring group can still catch two missing wedges...</pre>


<!-- Date: Tue Aug 2 12:16:31 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7152&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7153  width='300' align='top'></a></td><td>Bias current of IB2W0L4.  Got paged when it went pink.  Will increase the upper limit to keep it from going pink.</td></tr></table>
<!-- Author: Jen -->


<!-- Date: Tue Aug 2 14:58:24 2005 -->
<pre>Should check the following crowbars in the next access: 
LB0W0 L3bias1 has V=135 and I=0 
LB1W0 L3bias1 has V=134 and I=0 
LB1W1 L3bias1 has V=135 and I=0 
Reported with no physics by DS.</pre>
<!-- Author: Sebastian -->
<!-- Comment: Tue Aug  2 17;47;10 comment by...boveia -->
<pre>These look like they could have blown in the 3 house quench on 7/30.  They look biased in runs before this incident but after the kicker prefire on 7/28.</pre>


<!-- Date: Tue Aug 2 18:03:36 2005 -->
<table><tr><td><a href=/elog/notebooks/silicon/Tue_Aug__2_2005_180324.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7155  align='top'></a></td><td>Very long Scan Busy signal with the new EVB.</td></tr></table>
<!-- Author: Sebastian -->
<!-- Comment: Tue Aug  2 18;17;55 comment by...SCN/Markus/Sebastian -->
Looks like the changes Markus did to the EVB fixed the problem related to the two svx VRBs with only one wedge. However, after running for some 30k events (svx+isl) we got BTOs. No VRB was holding SCBS high because we found that after a very long scan busy (>1s) RC pulls BTO but the line goes low afterwards. We don't know why we have these long SCAN busy signals, but after we get the first we keep getting them (not only form b0svx06). With SVX alone we were able to run for 100k event with no BTO. 


<!-- Comment: Tue Aug  2 21;43;29 comment by...Sebastian -->
With ISL alone we first enconter BTO after a few hundred events (from b0svx06), but this time one could see the VRB that pulled busy high (svx07, slot 16 and 12). This would persist after each HRR. We started a new run and this time we ran for 40k events with no errors, but had to stop because the store was going to be dropped.
<!-- Date: Tue Aug 2 19:08:29 2005 -->
Did a coldstart and ran silicon for a while after Markus finished his EVB tests; everything looks good.  TeV just cleanly aborted store 4313.<br>
<!-- Author: Jen -->


<!-- Date: Tue Aug 2 20:59:46 2005 -->
<table border=0><td><tr><td bgcolor=#e0e030><h2> 
I am back and relieve Sebastian as SPL on-call 
</h2></td></tr></td></table>
<!-- Author: Marcel -->



<!-- Date: Tue Aug 2 22:56:15 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7156&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7157  width='300' align='top'></a></td><td>SVT SpyMon plots for store 4315, initial lum ~ 110.  Everything looks good in SVXMon too.</td></tr></table>
<!-- Author: Jen -->


<!-- Date: Wed Aug 3 01:33:58 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7158&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7158  width='300' align='top'></a></td><td>Changes in scalermon increased the L1 and L2 rates. At 90E30 the L1 rate is now 23kHz instead of the usual 20KHz. Got only one resonance this far.</td></tr></table>
<!-- Author: Sebastian -->


<!-- Date: Wed Aug 3 07:48:08 2005 -->
<a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&action=view&page=643&frame=2&anchor=03:35:01.568'>Aces hockerized Crate 13</a> at 3:30 this morning.  That's SE Bot Top.  Svxinter says, 
<pre> 
 Caenet_buffer: Error number 7 7 received 
group_info:cr_n = 13, response = 7 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 13, response = -251 
 Caenet_write: Error number -1 ffffffff received 
group_info:cr_n = 13, response = -1 
 Caenet_write: Error number -1 ffffffff received 
group_info:cr_n = 13, response = -1 
WARNING: found a power supply in the crate, which is not connected to a wedge in the detector, or absent in CAEN.map,  cr_n = 13, bd_n = 9 
 Caenet_write: Error number -253 received 
</pre> 
The last statement is true, if there is a power supply in Slot 9, it's not connected to a wedge.<br>
<!-- Author: Jen -->


<!-- Date: Wed Aug 3 08:37:02 2005 -->
<h3>Readout time of f223</h3> 
 
<b>Before changing ISL bits (run 202041):</b><br> 
HDI  rphiMean rphiRMS totalMean totalRMS<br> 
f223 5.13481 1.05955 23.6451 3.70734<br> 
<br> 
<b>After changing ISL bits (run 202081):</b><br> 
HDI  rphiMean rphiRMS totalMean totalRMS<br> 
f223 8.30272 1.17034 16.6717 3.01474<br> 
<br> 
Some improvement, but not much.  Still has the longest readout time for any ladder.<br>  
<!-- Author: Jen -->


<!-- Date: Wed Aug 3 13:18:13 2005 -->
<center><a href='/elog/notebooks/silicon/Wed_Aug__3_2005_131750.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7160' ></a><a href='/elog/notebooks/silicon/Wed_Aug__3_2005_131800.ps'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7162' ></a></center>
<!-- Author: SCN/Marcel -->
<!-- Comment: Wed Aug  3 13;23;19 comment by...MArcel/SCN -->
ISL SCANBUSY Signals, triggered on L2A, if SCANBUSY is already high (left) or low (right). More conclusions to follow

<!-- Comment: Wed Aug  3 14;37;23 comment by...SCN, Marcel, Sebastian etc etc  -->
<h3>Scan BUSY signals</h3>
Still trying to follow up on why there might be trouble with BUSY timeouts when using ISL.  The plots are examples of what is going on.  < ul>
<li> On the left, you have the READBUSY and SCANBUSY from ISL SRC and from the VRB daisy chain of L00 and ISL West. As is
evident from the plot, VRBSTA 0 = READBUSY and VRBSTA1 = SCANBUSY.
Also the measurement is of the SCANBUSY HIGH.  The Minimum time SCANBUSY is high is 65 &mu;s, Max is 6 ms, and average is 603.7 &mu;s.  You can see there is a big RMS, which is explained (to some extent) in a bit
<li> On the right, we've added the  SCANBUSY from SVXII (SBSV2) and the L2A from both SRCs.  The SVXII SCANBUSY behaves  differently, having a Minimum of 23 &mu;s, Maximum of 82 &mu;s and average of 42 &mu;s.
</ul>
The difference has to do with the way SCANBUSY works.  SCANBUSY goes high when the VRB starts moving data from all the channels into the output fifo, and doesn't go low until that transaction is finished.  If the output fifo gets full while the transaction is still happening, the transaction just waits, and SCANBUSY stays high.  So the characteristics of SCAN BUSY are determined by the amount of data in each VRB relative to the output fifo size (16 kB), as well as the L2A rate which commences filling the output fifo and the EVB bandwidth/rate, which empties it.<P>
From <a href="http://www-cdf.fnal.gov/htbin/twiki/bin/view/Main/VrbBalancing">Occupancy studies</a> we know that an SVXII Vrb averages around 700 Bytes/event, while ISL is around 1200 and L00 is 4600.  This difference means <ol>
<li> an SVXII Vrb can hold ~20 events in the output fifo, while L00 can only hold 3
<li> The EVB reads an event out of a SVXII crate ~6 times faster than out of the L00 crate. <P>
In addition, there is a potential difference for when SCANBUSY goes high.  FOr SVXII, SCANBUSY does not go high until after the TS sends the L2A to the Front End and to the EVB, whereas for 
ISL/L00, operating in Tracer Mode, as soon as the Express L2A is sent out, the data is digitized and readout (in ~ 28 &mu;s) and then SCANBUSY goes high.  The difference between the normal L2A and Express L2A is that normal L2A (which is all we have on the LA pictures) can be held up by processing of previous events in a fifo in the TS/TM, while express L2A cannot.  So when there
is moderately buffer usage the SCANBUSY from ISL/L00 will get a headstart of ~ 500 &mu;s relative to the SVXII and (current) EVB.<P>
All that being said, it appears to use that what is happening is that normally there is room in the output fifo for the event, so it just gets put in and SCANBUSY goes up and down on the order of  
eventsize/50 MB/sec (~ 15 &mu;s SVXII, 90 &mu;s L00) but if the output fifo is full or becomes full, you then have to wait for the EVB to drain it, which is on the timescale of a few ms, given possible processing of previous L2A and the fact that it has to drain all VRBs in a crate before SCANBUSY will go down.  Since L00 has large events sizes per VRB it is more likely to end up with in the full fifo scenario.   To test that, we studied the time between the previous L2A and the current L2A when the current L2A arrives while the ISL is still SCANBUSY or when it is  idle.  The results were:
<table>
<tr> <th> Scenario <th> Min &Delta;t between L2A <th> Max &Delta;t <th> Average &Delta;t </tr>
<tr> <td> SCANBUSY high <td> 470 &mu;s <td> 3 ms <td> 1.16 ms </tr>
<tr> <td> SCANBUSY low <td> 494 &mu;s <td> 32 ms <td> 8 ms </tr>
</table>
So, the Minimum is just the minimum time between L2A, i.e. the typical DONE handshake time.  More important, the average time between L2A when the second one is received while still processing the previous one is 1.16 ms &rarr; 860 Hz while when the second one is received while idle it is 8 ms &rarr; 125 Hz.  Since we know the current EVB hits a wall at ~ 400 Hz, this behaviour isn't surprising, and it jives with what we see 
<h3>"So what?"</h3>
Yes, this really doesn't explain the BUSY timeouts with the new eventbuilder experienced yesterday, but it did help a bit:
<ol>
<li> It gave us a clue about what to expect in terms of SCANBUSY times in ISL and SVXII
<li> In looking at this I was reminded of "transitional mode" of Logic analysers, which only records state changes, rather than a snapshot every <i>x</i> ns.  This is a way to stretch your memory  in a logic analyzer to look for timeouts of order 1 s while maintaining resonable edge resolution.  It should be relieve one of the limitations we suffered from yesterday.
</ol>
The funny thing is that what you expect from the EVB upgrade
<ol>
<li> Faster readout overall &rarr; faster output fifo draining.
<li>  Since it polls (rather than waiting for the L2A) the "headstart" from the express L2A shouldn't matter anymore.
</ol> Both of these things should shorten the SCANBUSY high time we think...

<!-- Comment: Wed Aug  3 14;38;15 comment by...  -->
</ol>

<!-- Comment: Wed Aug  3 15;03;52 comment by...SCN -->
Note that the SCANBUSY time (&gt; 900 &mu;s) <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=260&frame=2&anchor=18:03:36'>
seen yesterday</a> was for SVXII, which in today's measurement had a max of a factor of 10 less.  So it really seems like something is going wrong with the new EVB.


<!-- Date: Thu Aug 4 02:25:57 2005 -->
<h4> Clock trip fun </h4> 
We had a clock error and I saw the SVX turning yellow. So I quickly config'ed everything again, which brought everything back to normal. But it was pure coincidence that I was around. Steve Hahan is still investigating.
<!-- Author: Marcel -->


<!-- Date: Thu Aug 4 06:50:13 2005 -->
<h2>L00 Crowbar Access</h2> 
Went in and replaced the crowbars for LB0W0, LB1W0, and LB1W1.  All three crowbars had BIAS1 L3 channel blown.<br> 
Also checked BIAS1 on f223.  At standby, the power supply really was putting out 2V on that channel.  So it looks like the problem is a blown bias line for the ladder, not a power supply problem (for once).<br>  
<!-- Author: Jen & Marcel -->


<!-- Date: Thu Aug 4 07:37:00 2005 -->
<table border=0><td><tr><td bgcolor=#e0e030><h3>Tripped SVX, ISL and L00</h3> 
 
Due to error when preparing to add glycol to ISL (wrong valve oppened). Marcel and Jen are in the collision hall checking the PS for e3a4 which started tripping on DVDD after they were done with changing the L00 crowbars... Nice way to start the day!<br></td></tr></td></table>
<!-- Author: Marcel/Jen/Ulrich/Sebastian -->
<!-- Comment: Fri Aug  5 18;02;07 comment by...Chieftain &quot;Scares the Aces&quot; -->
that should be Chieftain "Messes with Valves"
<!-- Date: Thu Aug 4 08:11:34 2005 -->
<h3>Glycol in the ISL chiller II</h3> 
Today we started the second stage of reducing the freezing point of the ISL coolant. On June 22, we added some glycol to bring the glycol content to 7%. The final goal is 10%, equivalent to a freezing point of -3 C. We watch flows and pressures, as the 
viscosity of the coolant is increased. 
 
<p> 
Stefano and KC added 8 gallons of glycol to the ISL chiller. In the preparation of this work, a wrong valve was open, and air was sucked into the system. This caused a power supply trip on air separator overpressure. Adding the glycol (which has room temperature) caused another alarm due to high temperatures in the chiller. The alarm went away after a few seconds. We wait for about an hour for the coolant to mix.  
<!-- Author: Stefano, KC, Ulrich -->
<!-- Comment: Thu Aug  4 09;06;37 comment by...Ulrich -->
<br>
Stefano and KC measured the glycol concentration to about 12%. That means we slightly overshot the 10% we were aiming at and ended up with a freezing point of approximately -4 C. Eva and I will have another look at the system later to see if flows/temperatures are ok. All flows went down by about 0.1 lpm. We may have to open some supply valves a little bit to compensate for that.
<!-- Comment: Thu Aug  4 10;55;23 comment by...Eva, Ulrich -->
<br>
We took another look at the ISL flows and temperatures. Within 0.1 to 0.2 C, everything is back to where it was before adding the glycol. Since all flows went down by 0.1 lpm, we opened two supply valves a little more: ISL-CCE1 is now at 75% (was 70%), and ISL-PCE is at 80% (was 70%). We will check the system again in the evening.
<!-- Date: Thu Aug 4 08:21:17 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7163&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7164  width='300' align='top'></a></td><td>e3a4 has been tripping on DVDD since this morning. Jen and Marcel are in the collision hall now swapping PS cables with e5b...</td></tr></table>
<!-- Author: Sebastian/Chris -->
<!-- Comment: Thu Aug  4 08;57;48 comment by...Jen, Marcel, Sebastian, Chris -->
<h2>Access for e3a4</h2>
When we came back up from replacing the crowbars, we found that e3a4 had tripped overcurrent on DVDD.  We tried turning it back on, but it began tripping overcurrent on AVDD.  We powercycled the wedge, then rebooted the fib, then hockerized the crate; after each of these, it continued tripping overcurrent on AVDD.  We went back into the hall to try swapping cables, but were interrupted by the cooling trip.  When things came back up from the cooling trip, e3a4 no longer tripped... until 15 minutes later, when it went again, this time on DVDD overcurrent.<br>
So we returned to the hall and swapped cables with neighboring supply SB5W11.  After the cable swap, BOTH supplies held without tripping, for 5 minutes.  Then we swapped cables back, powered everything up... e3a4 has been holding since then (20 mins and counting).<br>
Conclusion:  probably not the power supply; if a power supply has gone bad, it usually stays bad, it doesn't stay on sometimes and trip off others.  The supply also shouldn't switch between tripping on AVDD and DVDD.  We'll see if it happens again, but it's hard to diagnose if we can't reproduce...<br>



<!-- Date: Thu Aug 4 09:12:30 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: SB3W10L4 
<li>problem n: 1500 
<li>Status: needs_observation
<li>Type: DVDD
<li>Action: Incident
<li>entered by: jpursley 
<li>description is: Supply began tripping on both DVDD and AVDD overcurrent during an access.  Tried swapping cables with SB5W11, but problem just disappeared.  e3a4 now holding fine.  More details in <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=260&frame=2&anchor=08:21:17'>elog</a>. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B3W10&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Thu Aug 4 09:35:08 2005 -->
<b>CAEN requests meeting</b>: Those of us dealing with CAEN met to discuss what more we want to request from them.  In addition to all the firmware, Rick will send an email to request that two more 400Hz SY527's, two more ISL supplies, and two more SVX supplies be built for us as spares.<br>
<!-- Author: Jen -->
<!-- Comment: Thu Aug  4 11;24;29 comment by...Sebastian -->
good... looks like we will need extra spares...


<!-- Date: Thu Aug 4 17:35:00 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7165&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7166  width='300' align='top'></a></td><td>SVT Spymon for store 4318.  Record initial lum = 139.  Everything looks good here and in SVXMon (deadtime with highlum table ~ 2% at start, L1A ~ 8kHz).</td></tr></table>
<!-- Author: Jen & Marcel -->


<!-- Date: Fri Aug 5 07:22:31 2005 -->
Got paged a couple times last night due to silicon resonance errors - it happened 5 times on the evening shift, twice more on the owl shift.  All had very low byte counts, and it looked like most repeat offenders were troubled ladders like e254.
<!-- Author: Jen -->



<!-- Date: Sat Aug 6 06:07:24 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7167&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7168  width='300' align='top'></a></td><td>SVT Spymon for store 4319.  Initial lum = 126.  Everything looks good in SVXMon.</td></tr></table>
<!-- Author: Jen -->


<!-- Date: Sat Aug 6 10:55:58 2005 -->
<pre>The three L00 ladders with blown crowbars (f803,fa03,fa13) now look biased in data taken after crowbar replacement.</pre>
<!-- Author: boveia -->

<!-- Date: Sat Aug 6 11:48:29 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: SB1W5L4 
<li>problem n: 11 
<li>Status: needs_work
<li>Type: DropReadOut_nonRX
<li>Action: Monitoring
<li>entered by: boveia 
<li>description is: Bit 1 high dropped from 100% to 10% of events after the kludge work, but unknown channel increased from 50% to 80%.  Comparing the ADC distributions with simulation, it looks like bit 7 low for 80% of events. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B1W5&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Sat Aug 6 11:50:01 2005 -->
<h2>Clock error</h2> 
Had a clock glitch while taking data.  Everything has been put in end-of-store condition (except CLC), and I have the silicon crates waiting in coldstart.  Steve is on his way in to take a look before clearing the errors.<br>
<!-- Author: Jen -->
<!-- Comment: Sat Aug  6 12;27;34 comment by...Jen -->
Steve arrived, reset the clock errors, and started playing with things.  all of SVX just tripped from standby.  L00 and ISL are still ok.
<!-- Comment: Sat Aug  6 12;36;22 comment by...Jen -->
Turned all of silicon OFF;  Steve says the clock error returns as soon as he clears it.
<!-- Date: Sat Aug 6 11:56:17 2005 -->
<pre> 
Event Number: 674 
<font color=yellow><B>10</B></font><font color=yellow><B>50</B></font><font color=blue>0d</B></font><font color=cyan><B>0b</B></font> <font color=yellow><B>04</B></font><font color=yellow><B>10</B></font><font color=yellow><B>41</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>08</B></font><font color=yellow><B>16</B></font><font color=yellow><B>08</B></font><font color=yellow><B>16</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font>  
<font color=black>e1</B></font><font color=black>54</B></font><font color=green>82</B></font><font color=cyan>fc</B></font> <font color=green><B>8d</B></font><font color=black>06</B></font><font color=blue><B>02</B></font><font color=black>02</B></font> <font color=blue><B>7f</B></font><font color=black>f3</B></font><font color=green><B>8c</B></font><font color=red><b>03</B></font> <font color=blue><B>3b</B></font><font color=black>f3</B></font><font color=red><b>04</B></font><font color=black>02</B></font> <font color=red><b>8e</B></font><font color=red><b>07</B></font><font color=blue><B>3b</B></font><font color=black>f3</B></font> <font color=red><b>04</B></font><font color=black>03</B></font><font color=red><b>8c</B></font><font color=red><b>45</B></font> <font color=blue><B>1c</B></font><font color=black>7c</B></font><font color=red><b>1a</B></font><font color=black>6a</B></font> <font color=blue><B>1e</B></font><font color=black>08</B></font><font color=blue><B>24</B></font><font color=black>0a</B></font> <font color=red>140ns</font> 
<font color=blue><B>25</B></font><font color=black>0b</B></font><font color=red><b>20</B></font><font color=black>0a</B></font> <font color=blue><B>25</B></font><font color=black>05</B></font><font color=blue><B>3e</B></font><font color=black>07</B></font> <font color=red><b>04</B></font><font color=black>ef</B></font><font color=blue><B>0e</B></font><font color=black>1b</B></font> <font color=red><b>0a</B></font><font color=black>1a</B></font><font color=red><b>09</B></font><font color=black>38</B></font> <font color=red><b>04</B></font><font color=black>32</B></font><font color=blue><B>2f</B></font><font color=black>33</B></font> <font color=red><b>0a</B></font><font color=black>32</B></font><font color=blue><B>0e</B></font><font color=black>37</B></font> <font color=red><b>0d</B></font><font color=black>36</B></font><font color=blue><B>28</B></font><font color=black>33</B></font> <font color=red><b>07</B></font><font color=black>32</B></font><font color=blue><B>2a</B></font><font color=black>27</B></font> <font color=red>600ns</font> 
<font color=red><b>04</B></font><font color=black>7a</B></font><font color=blue><B>3e</B></font><font color=black>03</B></font> <font color=red><b>00</B></font><font color=black>ef</B></font><font color=blue><B>04</B></font><font color=black>65</B></font> <font color=blue><B>2a</B></font><font color=black>03</B></font><font color=red><b>04</B></font><font color=black>f0</B></font> <font color=red><b>84</B></font><font color=red><b>66</B></font><font color=blue><B>65</B></font><font color=black>45</B></font> <font color=red><b>23</B></font><font color=black>4f</B></font><font color=blue><B>2c</B></font><font color=black>4e</B></font> <font color=blue><B>2d</B></font><font color=black>4c</B></font><font color=blue><B>6f</B></font><font color=black>5a</B></font> <font color=red><b>6c</B></font><font color=black>07</B></font><font color=red><b>07</B></font><font color=black>07</B></font> <font color=red><b>a6</B></font><font color=red><b>05</B></font><font color=blue><B>7b</B></font><font color=black>f3</B></font> <font color=red>1200ns</font> 
<font color=red><b>07</B></font><font color=black>07</B></font><font color=green><B>a5</B></font><font color=black>06</B></font> <font color=blue><B>7b</B></font><font color=black>04</B></font><font color=red><b>01</B></font><font color=black>f3</B></font> <font color=green><B>a4</B></font><font color=red><b>07</B></font><font color=blue><B>7b</B></font><font color=black>f3</B></font> <font color=red><b>07</B></font><font color=black>07</B></font><font color=red><b>a7</B></font><font color=red><b>04</B></font> <font color=blue><B>3a</B></font><font color=black>06</B></font><font color=blue><B>3b</B></font><font color=black>1b</B></font> <font color=blue><B>40</B></font><font color=black>1a</B></font><font color=blue><B>41</B></font><font color=black>0f</B></font> <font color=blue><B>46</B></font><font color=black>0b</B></font><font color=blue><B>47</B></font><font color=black>05</B></font> <font color=blue><B>7b</B></font><font color=black>f3</B></font><font color=red><b>07</B></font><font color=black>07</B></font> <font color=red>1840ns</font> 
<font color=green><B>a6</B></font><font color=red><b>05</B></font><font color=blue><B>2b</B></font><font color=black>05</B></font> <font color=red><b>2b</B></font><font color=black>0e</B></font><font color=blue><B>2c</B></font><font color=black>1b</B></font> <font color=blue><B>2d</B></font><font color=black>36</B></font><font color=blue><B>2e</B></font><font color=black>2f</B></font> <font color=blue><B>2f</B></font><font color=black>15</B></font><font color=blue><B>34</B></font><font color=black>07</B></font> <font color=blue><B>7b</B></font><font color=black>f3</B></font><font color=red><b>07</B></font><font color=black>07</B></font> <font color=green><B>a5</B></font><font color=black>06</B></font><font color=blue><B>7b</B></font><font color=black>f3</B></font> <font color=red><b>07</B></font><font color=black>07</B></font><font color=green><B>a4</B></font><font color=red><b>07</B></font> <font color=blue><B>53</B></font><font color=black>05</B></font><font color=red><b>53</B></font><font color=black>2a</B></font> <font color=red>2500ns</font> 
<font color=red><b>52</B></font><font color=black>19</B></font><font color=blue><B>53</B></font><font color=black>04</B></font> <font color=blue><B>7b</B></font><font color=black>f3</B></font><font color=red><b>07</B></font><font color=black>1a</B></font> <font color=blue><B>17</B></font><font color=black>15</B></font><font color=blue><B>5f</B></font><font color=black>6a</B></font> <font color=red><b>5f</B></font><font color=black>6a</B></font><font color=red><b>5f</B></font><font color=black>6a</B></font> <font color=red><b>5f</B></font><font color=black>6a</B></font><font color=red><b>57</B></font><font color=black>c2</B></font> <font color=red><b>c2</B></font><font color=black>c4</B></font><font color=red><b>00</B></font><font color=black>00</B></font>                   <font color=red>3120ns</font> 
</pre>
<!-- Author: boveia -->
<!-- Comment: Sat Aug  6 11;59;10 comment by...boveia -->
e154 recently.  Despite my comment about the ADC distributions, the data look like they have a lot more than just a bit 7 problem.

<!-- Date: Sat Aug 6 12:02:15 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: SB1W5L4 
<li>problem n: 11 
<li>Status: needs_work
<li>Type: DropReadOut_nonRX
<li>Action: Monitoring
<li>entered by: boveia 
<li>description is: Forget about bit 7.  The <a href= 'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=260&frame=2&anchor=11:56:17'> raw data</a> is 'bananas.' 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B1W5&layer=L4">report</a></ul>
<!-- Author: Problem DB -->





<!-- Date: Sat Aug 6 12:42:59 2005 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7169&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7169' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7170&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7170' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7171&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7171' width='300'></a></center>
<!-- Author: Jen -->
<!-- Comment: Sat Aug  6 12;43;39 comment by...Jen -->
State of the silicon:  SVX tripped off, ISL and L00 were turned off to avoid tripping.

<!-- Comment: Sat Aug  6 13;44;32 comment by...Jen -->
Looks like we added the glycol just in time.  ISL portcard temps dropped about 2 degrees (from 9 to 7.3) right after I shut everything down, and have been steady since then.  Silicon is still off, the Steves are still trying to figure out the clock problem.

<!-- Comment: Sat Aug  6 14;49;22 comment by...Ulrich H. -->
Freezing portcard cooling lines is not a danger here. The only situation in which a PC line could freeze is: flow from SVX chiller, but no flow from the ISL chiller to the SVX portcards. The temperature drop you see is probably just the missing heat load. Anyway, it is nice to have the glycol.

<!-- Comment: Sat Aug  6 15;19;27 comment by...Jen -->
Clock problem remains undiagnosed.  Silicon off.  I've asked to be paged when the clock is back up and has been error free for an hour.


<!-- Date: Sat Aug 6 18:09:44 2005 -->
<h2>Turning the silicon back on...</h2> 
No clock errors for almost three hours, so I'm bringing the silicon back to life.  L00 is on.  PS Gui just died, so I'm restarting it.<br>
<!-- Author: Jen -->
<!-- Comment: Sat Aug  6 18;58;16 comment by...Jen & Ankush -->
Ok, everything is back to standby, except for IB5W5L4, which trips AVDD Overcurrent.  So far we've tried rebooting the fib, rebooting svxinter, and are now hockerizing crate 16.  

<!-- Comment: Sat Aug  6 19;02;26 comment by...Jen & Ankush -->
Hockerizing worked!  IB5W5L4 is now holding.  We're running a coldstart now, then we'll bring up the bias.<br>


<!-- Date: Sat Aug 6 19:56:35 2005 -->
<h4> Fun with e554 </h4> 
This number seems to be cursed ...<br> 
That is what we get from it:<br> 
<pre> 
e554c1c1 8d9c88fc  
8c9c8080 baacb980 8b9cbb81 899c8a9c  
879c889c b189b081 a69cb280 a49ca59c  
a29ca39c a09ca19c c2c2fff0 88fce560 
</pre>
<!-- Author: Marcel -->
<!-- Comment: Sat Aug  6 20;24;31 comment by...Jen, Ankush, Marcel -->
after several attempts to reset the fib, reset the VRB, etc. we still get:
<pre>
c1c17f03 1bfee554 
80808dab b9808cab bb82baad 8aab8bab 
988289ab 9ae399a5 88ab9b84 8a8387ab 
8c948b89 a5828d83 a7b7a6b9 a6aba880 
8b8f8a84 8d838c8d a4aba5ab a2aba3ab 
98899785 9a9b99d0 dcf19b86 def1ddef 
a0aba1ab 93b29283 b483948c b6ecb589 
b880b78b c2c2fff0
</pre>
So we need to have a look at it in quiet time, to see, what is really going on. I suspect the FIB.


<!-- Comment: Sat Aug  6 20;41;31 comment by...Ankush, Jen and Marcel -->
<h3>Problem with the fib</h3>

<p>We quickly ran this ladder with SVXDAQ confirmed e554 was giving garbage data.  We then swapped the fibres to the VRB and the problem followed the fibres.  SO we confirmed the VRB is fine.</p>


<p>The problem clearly lies in the collision hall.  The most likely cuplrit is the fib.  During the next access we should swap the fib and see if it fixes the problems.</p>
<!-- Comment: Sat Aug  6 20;48;26 comment by...Ankush, Jen and Marcel -->
<p>...and this ladder had been taken out of the hardware database for the time being until we fix the FIB.</p>
<!-- Date: Sat Aug 6 20:38:30 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: SB5W4/5All 
<li>problem n: 1501 
<li>Status: needs_access_2hr
<li>Type: Chip_bit_error
<li>Action: Diagnosis
<li>entered by: amitra 
<li>description is: <p>This ladder is giving no data.  Came to the conclusion there is an error with the fib. During next access, we should swap this ladder's fib.</p> 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B5W4/5&layer=All">report</a></ul>
<!-- Author: Problem DB -->




<!-- Date: Sat Aug 6 20:50:37 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: SB5W5L4 
<li>problem n: 1502 
<li>Status: needs_work
<li>Type: Unknown
<li>Action: Removed_hdwdb
<li>entered by: amitra 
<li>description is: <p>Due to problems with the FIB this ladder has been taken out of the hardware database.  When we fix the FIB, this ladder will be put back into the HWDB.</p> 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B5W5&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Sun Aug 7 08:00:36 2005 -->
<h3>Hockerized crate 9</h3> 
While I happened to be in the control room, crate 9 (NE Top Top) lost communication, and I got to show the new ace how to hockerize.  Svxinter says, 
<pre> 
 Caenet_write: Error number -1 ffffffff received 
group_info:cr_n = 9, response = -1 
 Caenet_write: Error number -2 fffffffe received 
group_info:cr_n = 9, response = -2 
</pre> 
This would have been a confusing one for an ace to diagnose, as the alarms page just said that one or more power supplies was off.  But the power supplies were gray in PSGui, and the wedges showed up blue in Imon.<br>
<!-- Author: Jen -->

<!-- Date: Sun Aug 7 11:47:50 2005 -->
<p><font size=+1> D-mode Calibration Job Started </font>
<p><pre>
Job id            : 417
Calibration type  : standard
Run period tag    : Aug_2005
DPS-off run       : 202372
DPS-on run        : 202371
Requested by      : sasha baroiant
Main process id   : 8148
Comment           : hrr during dps ON mode due to b0svx06 error: 6 L2 w/o L1. Run continued and ended properly.
</pre>
<!-- Author: CDF silicon D-mode calibration job server v1.4 -->



<!-- Date: Sun Aug 7 14:47:46 2005 -->
<h4>Had another clock error...</h4> 
earlier today (~9am).  But it cleared on the first try and hasn't happened again... yet.  See <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&action=view&page=656&frame=2&anchor=09:13:13.327'>Steve's entry in the main elog</a>.<br>
<!-- Author: Jen -->


<!-- Date: Sun Aug 7 15:02:32 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7172&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7173  width='300' align='top'></a></td><td>SVT Spymon plot for store 4320.  Initial lum = 120, but a L3 converter died and it took a while to start a run.  When the run started, we had one ISL fib go red in SVXMon, but an HRR brought it back.  Everything now looks good here and in SVXMon.</td></tr></table>
<!-- Author: Jen -->


<!-- Date: Sun Aug 7 17:58:22 2005 -->
<h3>More clock errors...</h3> 
So the clock has begun erring frequently again; at first it was ~20 mins apart, then it stretched out to almost an hour apart, now it's beginning to be about 15 mins apart.  So far no one has been able to reach Steve Chappa to come help swap out the clock power supply, which is the next best idea to curing this.  Until he can be reached, the plan is to hobble along as best as possible.  But if the frequency of errors continues to be around 15-20 minutes, we'll take out silicon and put it to standby.  No point in using it for 10 min runs that no one will look at anyway!<br>
<!-- Author: Jen & Marcel (remotely) -->
<!-- Comment: Sun Aug  7 18;09;49 comment by...Marcel -->
see <a href='http://www-cdfonline.fnal.gov/java/cdfdb/servlet/ClockErrors'> here </a>for the clock error frequency.
<!-- Date: Sun Aug 7 18:02:04 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: SB0W5All 
<li>problem n: 1491 
<li>Status: fixed
<li>Type: Other
<li>Action: Diagnosis_update
<li>entered by: boveia 
<li>description is: Silicon was ok.  SVT group traced this problem to their hardware. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B0W5&layer=All">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Sun Aug 7 18:13:59 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: IB5W5L2 
<li>problem n: 853 
<li>Status: needs_work
<li>Type: Chip_capid
<li>Action: Monitoring
<li>entered by: boveia 
<li>description is: Chip 98 has stuck cap id (=15) for every event since 6/16 or so, thus no data from this chip in ADCMon. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=ISL&wedge=B5W5&layer=L2">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Sun Aug 7 18:13:59 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: IB5W5L2 
<li>problem n: 1494 
<li>Status: fixed
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: boveia 
<li>description is: See thread 853. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=ISL&wedge=B5W5&layer=L2">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Sun Aug 7 18:20:55 2005 -->
<b>Software protection trip of SB3W2L0</b>:<br> 
Running with this clock we're getting lots of errors.  SB3W2L0 just tripped off the whole wedge on a software protection trip of DVDD. It measured the current to be 187.0 when the software limit is 150.0.  Brought the wedge back up with no problems, DVDD current is really about 83.<br>
<!-- Author: Jen -->

<!-- Date: Sun Aug 7 18:22:33 2005 -->
Entered in Silicon database: <ul>
<li>Ladder: SB0W9L2 
<li>problem n: 1492 
<li>Status: fixed
<li>Type: Chip_bit_error
<li>Action: Monitoring
<li>entered by: boveia 
<li>description is: Data were mostly garbage from runs 200725 through 200756 (I couldn't verify bit 7 high from ADC distributions or raw data.)  Since then, no errors. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B0W9&layer=L2">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Sun Aug 7 18:22:38 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7174&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7175  width='300' align='top'></a></td><td>History of SB3W2L0 DVDD.</td></tr></table>
<!-- Author: Jen -->


<!-- Date: Sun Aug 7 18:24:10 2005 -->
<pre>---- Monitoring Summary ----   
 
NEW PROBLEMS DISCOVERED:   
 
None? 
 
"FIXED" PROBLEMS:   
 
LB0W0L3 bias1, LB1W0L3 bias 1, LB1W1L3 bias1 - Blown crowbars in 7/30 3-house quench.  Fixed and data now look biased. 
 
OLD PROBLEMS: 
 
SB0W9L2 - Garbage data for a few runs.  Looks all better for now. 
IB5W5L2 - Chip 98 has cap id stuck at 15 (old problem.)  Passed to Detector group. 
 
LB0W5L3 (f853)  - Chip 2 possibly unbiased.  I need to look at this.  
SB2W10L4(e2a4)  - Long readout time.  Need to look at bias scans.</pre>
<!-- Author: boveia -->


<!-- Date: Sun Aug 7 19:35:47 2005 -->
<h3>Update of clock situation:</h3> 
Clock hasn't erred in an hour and a half.  We eventually got hold of Steve Chappa, but he wants to wait until we see two errors right in a row again before trying to swap out the clock power supply.<br> 
<b>Plan:</b>  if we get another clock error, shift crew will page me AND the ops manager for permission to clear the errors and start a new run with silicon.  If they get 2 errors within 15 minutes of each other, they'll page ops, who will get Steve Chappa and Dervin in to swap the supply, and they'll page me to come turn off silicon while the supply is being swapped out.<br>
<!-- Author: Jen -->


<!-- Date: Sun Aug 7 20:27:14 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7176&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7176  width='300' align='top'></a></td><td>Resonance Plots</td></tr></table>
<!-- Author: Marcel -->
<!-- Comment: Sun Aug  7 20;38;40 comment by...Marcel -->
Some studies on our resonance problems. The four plots show number of resonances  versus L1A rate, Luminosity, Deadtime and a Lumi/L1A plot starting with run 190000<br>
One thing is clear, we do not see any resonances below 13 KHz, which is not really surprising because we are not running below 15 kHz anymore and  we have plenty of time at 15 kHZ L1A rate anyway. It really starts at around 19/20 KHz, because that is the trigger rate range, the Scaler try to keep in, if I remember right. Luminositywise, it is not so clear, the peak at 40 is from DPS kicking in, the peak at approx 90E30 is from probably switching tables. No clear correlation with deadtime.<br>
We now know, we have a long tail up to roughly 40 microseconds. So nothing really surprising here, you expect more resonances when approaching this region. 

<!-- Date: Sun Aug 7 21:03:16 2005 -->
<p><font size=+1> D-mode Calibration Job Finished </font>
<p><pre>
This is a computer-generated silicon D-mode calibration status report.
Silicon D-mode calibration job 417 FAILED.

Job info
--------
Calibration type : standard
Beam condition   : on
DPS-off run      : 202372
DPS-on run       : 202371
Requested by     : sasha baroiant
Submitted        : 2005-08-07 11:46:09
Completed        : 2005-08-07 21:02:50
Status           : FAILURE
Comment          : hrr during dps ON mode due to b0svx06 error: 6 L2 w/o L1. Run continued and ended properly.

Last processing stage reported : updating ped tables with fibkill channels
Failure diagnostics :
child process exited abnormally
</pre>
<!-- Author: D-mode script  -->
<!-- Comment: Mon Aug  8 22;00;06 comment by...Jason -->
One of the silicon configuration servers was not responding, so I restarted it.
I finished this calibration, and the backlog of runs without used sets was cleared 
within 5 minutes.
<BR>
Ordinarily this configuration server would be restarted when the used sets maker is restarted, but since the used sets maker itself was fine, the restart never happened.
I am putting in a check now (as cron job running under cdfopr on fcdflnx3) which checks that the silion configuration server on fcdflnx3 is up and responding to queries.  If it is not, then the used set maker will be terminated and restarted.
<!-- Date: Sun Aug 7 21:03:16 2005 -->
<p><font size=+1> D-mode Calibration Job Finished </font>
<p><pre>
This is a computer-generated silicon D-mode calibration status report.
Silicon D-mode calibration job 417 FAILED.

Job info
--------
Calibration type : standard
Beam condition   : on
DPS-off run      : 202372
DPS-on run       : 202371
Requested by     : sasha baroiant
Submitted        : 2005-08-07 11:46:09
Completed        : 2005-08-07 21:02:50
Status           : FAILURE
Comment          : hrr during dps ON mode due to b0svx06 error: 6 L2 w/o L1. Run continued and ended properly.

Last processing stage reported : updating ped tables with fibkill channels
Failure diagnostics :
child process exited abnormally
</pre>
<!-- Author: D-mode script  -->



<!-- Date: Mon Aug 8 00:27:08 2005 -->
<h4>Hockerized crate 6 (SW Bot Bot)</h4> 
So the aces paged me with a mysterious silicon inhibit from RC.  There was nothing showing wrong in IMON or PSGui, so I came in.  When I got here, I still couldn't see anything wrong.  I paged Marcel, who pointed me to the silicon inhibits page from IFix.  That showed the problem was with SB1W8L0.  I checked that wedge, and sure enough, L0 was showing a voltage of only 86 V when the setting was for 104 V.  Tried powercycling the wedge, but that did nothing.  So then we hockerized crate 6.  When I brought the wedges back up, SB1W8L0 was fine.  Since we'd had a clock error right before hockerizing, I stole the crates to do a coldstart and get things through config.  Then I let the aces start another run.<br> 
We just had another clock error, 8 minutes after they reset the other error.  Now paging ops manager to see if they want to swap out the clock power supply.<br>
<!-- Author: Jen & Marcel (remotely) -->


<!-- Date: Mon Aug 8 01:04:15 2005 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7177&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7177' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7178&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7178' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7179&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-7179' width='300'></a></center>
<!-- Author: Jen -->
<!-- Comment: Mon Aug  8 01;06;21 comment by...Jen -->
Clock error.  Ace tried to end run right away, pcal crates got stuck, while she was shepherding everything tripped.

<!-- Comment: Mon Aug  8 01;17;03 comment by...Jen & Marcel -->
Put all the surviving wedges to standby; now awaiting Steve Chappa's arrival before deciding what to do next.

<!-- Comment: Mon Aug  8 01;43;53 comment by...Jen -->
successions of clock errors... the last one just tripped off the remainder of SVX.  Still no sign of Chappa.

<!-- Comment: Mon Aug  8 02;07;07 comment by...Jen -->
Chappa and Dervin are here, they're swapping out the clock power supply.  All the rest of the silicon has been turned to OFF.


<!-- Date: Mon Aug 8 02:09:42 2005 -->
Also, heard from RC that b0fib06 had stopped updating.   Tried to ping it and couldn't.  So I shepherded it from RC, it is now pingable.
<!-- Author: Jen -->


<!-- Date: Mon Aug 8 06:57:03 2005 -->
<h3>Bringing the silicon back up</h3> 
No clock errors since 4am.  Bringing the silicon back, L00 is up so far.<br>
<!-- Author: Jen -->
<!-- Comment: Mon Aug  8 07;21;36 comment by...Jen -->
Ok, everything is at standby and running in coldstart.  Had some problems with tripping (particularly SB1W2L4 going DVDD Overcurrent) but everything seems to be staying now.  After coldstart, will try biasing.

<!-- Comment: Mon Aug  8 07;35;24 comment by...Jen -->
Silicon is successfully biased and through coldstart.  Now starting a physics run with silicon.

<!-- Comment: Mon Aug  8 07;45;29 comment by...Jen -->
aaaaannnndddd everything looks good in SVT Spymon and SVXMon.  Looks like we came out of this clock trip without any new damage.


<!-- Date: Mon Aug 8 09:01:29 2005 -->
<h4>Week summary</h4> 
<ul> 
<li> Store 4313 (Mon 14:00) 
   <ul> 
   <li> f223: set ISL bits to 1111 for chips 0-3 and 12-15; slightly reduced readout time 
   <li> VRB_32 in b0svx03 went into WYSIWIG mode; rebooted b0svx03 
   <li> Electron lens tripped off, came back on 
   <li> b0svx08 failed config and wouldn't abort; rebooted 
   </ul> 
<li> Store 4315 (Tues 23:00) 
   <ul> 
   <li> Aces hockerized crate 13 
   <li> one clock glitch, Marcel config'd in time to prevent trips 
   </ul> 
<li> Thu 5:30 
   <ul> 
   <li>Access to replace 3 L00 crowbars, blown in Sat's quench 
   <li> e3a4 began tripping and eventually stopped on its own 
   <li> Added glycol to ISL coolant, tripped detector 
   </ul> 
<li> Store 4318 (Thu 17:30) 
   <ul> 
   <li> Record luminosity 139 
   <li> Lots of resonance errors 
   </ul> 
<li> Store 4319 (Sat 6:00) 
   <ul> 
   <li> Repeated clock errors 3 hrs into store; eventually tripped off silicon 
   <li> Brought silicon back, e554 sees no data; requested access to swap fib and removed from hwdb 
   <li> Hockerized crate 9 
   <li> Took D-mode calibration 
   </ul> 
<li> Store 4320 (Sun 15:00) 
   <ul> 
   <li> Repeated clock errors again; eventually stop 
   <li> Hockerized crate 6 
   <li> Repeated clock errors again, this time trips silicon 
   <li> Bring silicon back up after swapping clock power supply 
   </ul> 
</ul> 
<!-- Author: Jen -->