<!-- Title: Day shift. Sat Jul 19, 2003 -->
<!-- SciCo: Joe Boudreau -->
<!-- DAQAce: Pierluigi Castastini -->
<!-- MonAce: Stan Forrester -->
<!-- CO: Rong Shyang Lu -->
<!-- OpManager: Masa Tanaka -->
<!-- Notes: -->
<pre>We inherit stable running conditions and a luminosity of  
20 E30.  </pre>


<!-- Date: Sat Jul 19 08:05:53.784 2003 -->
The reciver display of CSl window seems to be frozen. Closed and restarted.
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166369' target=_top>166369</a>) -->


<!-- Date: Sat Jul 19 08:21:02.545 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-8261&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-8261  width='300' align='top'></td><td>Off-diagonal components are seen. Notify the L2 expert.</a></td></tr></table>
<!-- Author: Rong-Shyang -->
<!-- Comment: Sat Jul 19 10;01;55 comment by...Nathan Goldschmidt -->
This is a long-standing, known problem. This error rate from CLIST is acceptable (nCluster mismatch rate < 10^-3). 
<!-- Date: Sat Jul 19 08:23:10.779 2003 -->
<pre>COT 
Done timeout. Auto HRR recovered 
 (MLE) b0cot09:Messenger:8:19:48 AM->Event 674771: Bunchcounters in slot 2 (BC=112) and slot 4 (BC=65) disagree (L2B=0)  
</pre>
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166369' target=_top>166369</a>) -->
<!-- Comment: Sat Jul 19 08;25;40 comment by...Pierluigi -->
<pre>Done timeout followed by FERMI_TDC_NOT_READY cause b0cot06.
REquest to shepherd the crate. Done and run recovered.</pre>

<!-- Comment: Sat Jul 19 08;50;41 comment by...Pierluigi -->
(MLE) b0cot19:Messenger:8:48:46 AM->Event 999467: Bunchcounters in slot 5 (BC=18) and slot 10 (BC=9) disagree (L2B=2)

<!-- Comment: Sat Jul 19 08;58;32 comment by...Pierluigi -->
(MLE) b0cot19:Messenger:8:57:00 AM->Event 1093047: Bunchcounters in slot 5 (BC=109) and slot 9 (BC=18) disagree (L2B=0)


<!-- Date: Sat Jul 19 09:06:13.050 2003 -->
<pre>PCAL 
Done timeout. Auto HRR recovered. 
(MLE) b0pcal06:Messenger:9:03:54 AM->Event 1170533: Bunchcounters in slot 15 (BC=0) and slot 17 (BC=24) disagree (L2B=1)</pre>
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166369' target=_top>166369</a>) -->


<!-- Date: Sat Jul 19 09:07:46.167 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8262&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8262' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8263&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8263' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8264&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8264' width='300'></a></center>
<!-- Author: Stan Forrester -->


<!-- Date: Sat Jul 19 09:16:27.541 2003 -->
<pre>A lot of L2 decision time out. 
 
Running Started 
 
Read L2 prescales 
SvtMon: SVT buffno 1 != event buffno 0 
SvtMonitor saw 5 sequential errors at Jul 19 08:20:02 2003 
FaultManager: Fatal error, sending L2 timeout (L2A 1257267) 
 
</pre>
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166369' target=_top>166369</a>) -->
<!-- Comment: Sat Jul 19 09;29;57 comment by...Pierluigi -->
<pre>again also after end run.
rror: Startload is still low. ModDone bits 0x4007fdff for event 83397
TIMEOUT: Mod done not set for slot 12 (0x0007fdff, cnt 1)

Error: Startload is still low. ModDone bits 0x4007fdff for event 101731
TIMEOUT: Mod done not set for slot 12 (0x0007fdff, cnt 2)

Error: Startload is still low. ModDone bits 0x4007fdff for event 155957
TIMEOUT: Mod done not set for slot 12 (0x0007fdff, cnt 3)
Slot12 saw 3 sequential errors at Jul 19 08:31:50 2003
FaultManager: Fatal error, sending L2 timeout (L2A 4685)

</pre>
<!-- Date: Sat Jul 19 09:18:53.704 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166369">166369</A>
 Terminated at 2003.07.19 09:17:51
<!-- Author: RunControl -->


<!-- Date: Sat Jul 19 09:20:23.021 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166369">166369</A>
TERMINATE: end run 
<!-- Author: Pierluigi* 2080 -->



<!-- Date: Sat Jul 19 09:21:21.980 2003 -->
MCR calls to say that they know what caused the 150 micron 
shift last night.. Additionally, they can put it back if we 
desire.  We ask them to keep it where it is.												
<!-- Author: boudreau -->

<!-- Date: Sat Jul 19 09:25:36.764 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166370">166370</A>
 Activated at 2003.07.19 09:23:51
<!-- Author: RunControl -->



<!-- Date: Sat Jul 19 09:34:14.044 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166370">166370</A>
 Terminated at 2003.07.19 09:31:28
<!-- Author: RunControl -->


<!-- Date: Sat Jul 19 09:34:16.310 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166370">166370</A>
TERMINATE: end run
<!-- Author: Pierluigi* 2080 -->


<!-- Date: Sat Jul 19 09:41:16.291 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371">166371</A>
 Activated at 2003.07.19 09:38:36
<!-- Author: RunControl -->


<!-- Date: Sat Jul 19 09:41:17.539 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371">166371</A>
ACTIVATE: start run 
<!-- Author: Pierluigi* 2080 -->


<!-- Date: Sat Jul 19 09:42:43.108 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166370">166370</A>
ACTIVATE: start run
<!-- Author: Pierluigi* 2080 -->
<!-- Comment: Sat Jul 19 09;50;59 comment by...Pierluigi -->
<pre>So, we are running...
after L2 decision timeouts, L2 and SVT experts were paged. L2 guy told us to end the run and restart again, if we got again in trouble the problem could be SVT. SVT guy told us to perform a check ("~ashmansk/bin/whodunnit") and it seemed to be ok. In the while we ended again the run (we had again the same problems) and paged again L2 guy. He asked to end again and restart (reset- partition-config-activate). THIS TIME IT WORKED!!  </pre>
<!-- Comment: Sat Jul 19 10;09;08 comment by...N. Goldschmidt -->
It seems that the first time the DAQ ace started a new run, he did so without partitioning. The next time, he did partition. This suggests to me that L2 is not the culprit. Am I wrong? 
<!-- Comment: Sat Jul 19 12;31;24 comment by...Saltzberg -->
<BR>
You are right.
<p>
Those error messages from L2 are well known since Sept. '01
(and known to a few of us before then). 
You get them because L2 is the first crate in the system
that performs this check on synchronization of the data in
this path.
Over the years, Matt Worcester has checked many such events offline that go to the error trigger  and they are indeed a problem upstream of the L2 crate.  
<!-- Date: Sat Jul 19 09:46:35.503 2003 -->
<pre>We lose about 27 minutes. 
 
* First we have level 2 timeout errors. We page L2 and SVT. 
* L2 tells us to restart the run, and see if errors persist. 
  If so==> probably SVT. 
* SVT experts then ask us to run some diagnostics and tell us 
  that SVT is OK. 
* We coldstart the run while waiting for L2 expertise to arrive 
* And that fixes the problem. 
</pre>
<!-- Author: Boudreau -->
<!-- Comment: Sat Jul 19 12;38;29 comment by...Saltzberg -->
If I understand their diagnostic correctly, the SVT diagnostic must be run while the system is in the "HALT" state just after one of these errors.   
<!-- Date: Sat Jul 19 10:04:41.882 2003 -->
<pre>CCAL 
Done timeout. auto HRR recovered. 
Error (25) >>  (MLE) b0ccal13:Messenger:10:00:59 AM->Event 216875: Bunchcounters in slot 17 (BC=109) and slot 18 (BC=59) disagree (L2B=1)</pre>
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->



<!-- Date: Sat Jul 19 10:14:20.215 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8265&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8265' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8266&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8266' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8267&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8267' width='300'></a></center>
<!-- Author: Stan Forrester -->


<!-- Date: Sat Jul 19 10:26:12.653 2003 -->
<pre>WCAL 
Done timeout. auto HRR recovered 
(MLE) b0wcal05:Messenger:10:23:53 AM->Event 463420: Bunchcounters in slot 16 (BC=80) and slot 20 (BC=127) disagree (L2B=1)</pre>
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->


<!-- Date: Sat Jul 19 10:36:23.868 2003 -->
<pre>COT  
done timeout. Auto HRR recovered 
(MLE) b0cot14:Messenger:10:34:13 AM->Event 572488: Bunchcounters in slot 5 (BC=6) and slot 6 (BC=0) disagree (L2B=3</pre>
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->
<!-- Comment: Sat Jul 19 11;30;39 comment by...Pierluigi -->
 (MLE) b0cot06:Messenger:11:28:28 AM->Event 1140063: Bunchcounters in slot 6 (BC=83) and slot 7 (BC=68) disagree (L2B=1)
<!-- Comment: Sat Jul 19 12;06;09 comment by...Pierluigi -->
(MLE) b0cot19:Messenger:12:04:28 PM->Event 1501309: Bunchcounters in slot 5 (BC=62) and slot 6 (BC=109) disagree (L2B=0)
<!-- Comment: Sat Jul 19 12;20;10 comment by...Pierluigi -->
(MLE) b0cot09:Messenger:12:18:22 PM->Event 1636511: Bunchcounters in slot 2 (BC=112) and slot 6 (BC=62) disagree (L2B=1)
<!-- Comment: Sat Jul 19 12;52;30 comment by...Pierluigi -->
<pre>several DT. FERMI_TDC, request to shepherd b0cot00. we did but It didn'nt recover and RC aked to shepherd again b0cot00. this time recovered</pre>
<!-- Comment: Sat Jul 19 12;57;17 comment by...Pierluigi -->
 (MLE) b0cot06:Messenger:12:55:30 PM->Event 1963941: Bunchcounters in slot 6 (BC=115) and slot 7 (BC=109) disagree (L2B=1)
<!-- Comment: Sat Jul 19 14;24;06 comment by...Pierluigi -->
(MLE) b0cot06:Messenger:2:22:19 PM->Event 2762907: Bunchcounters in slot 6 (BC=127) and slot 7 (BC=86) disagree (L2B=2)
<!-- Date: Sat Jul 19 10:50:14.053 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8268&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8268' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8269&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8269' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8270&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8270' width='300'></a></center>
<!-- Author: Stan Forrester -->



<!-- Date: Sat Jul 19 11:56:48.070 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8271&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8271' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8272&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8272' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8273&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8273' width='300'></a></center>
<!-- Author: Stan Forrester -->








<!-- Date: Sat Jul 19 13:11:21.218 2003 -->
<pre>We have an alarm on TSU BSU CSP.  The problem is the CCU, 
which supplies power to all of these.  However the expert 
tells us that this is a monitoring problem.			</pre>
<!-- Author: Boudreau :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->
<!-- Comment: Sat Jul 19 13;27;50 comment by...Boudreau -->
<pre>we are now loading new constants into the CCU.. This is possibly
changing voltages; note, we don't really know if we have a 
HV problem or a monitoring problem.  </pre>
<!-- Date: Sat Jul 19 13:26:20.773 2003 -->
GLOBAL ALARM - CSP, BSU and TSU.  Invesitgated 1RR15_C on first floor.  Found that nearly all involved ccu's (all except 3) had channel 1 voltage at 770.  Upon loading data file for each ccu found that proper voltages were either 875 or 900 volts.
<!-- Author: Stan Forrester :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->



<!-- Date: Sat Jul 19 13:43:19.360 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8274&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8274' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8275&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8275' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8276&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8276' width='300'></a></center>
<!-- Author: Stan Forrester -->


<!-- Date: Sat Jul 19 13:58:35.311 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8277&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8277' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8278&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8278' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8279&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8279' width='300'></a></center>
<!-- Author: Stan Forrester -->



<!-- Date: Sat Jul 19 15:10:08.323 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8280&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8280' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8281&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8281' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8282&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8282' width='300'></a></center>
<!-- Author: Stan Forrester -->


<!-- Date: Sat Jul 19 15:15:20.101 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8283&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8283' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8284&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8284' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8285&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8285' width='300'></a></center>
<!-- Author: Stan Forrester -->


<!-- Date: Sat Jul 19 15:23:19.281 2003 -->
BUSY timeout cause SCPU b0eb25 slot 10. auto HRR recovered.
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->


<!-- Date: Sat Jul 19 15:27:56.624 2003 -->
bmu alarm -- SW BMU SW #84 tripped.  Tried to turn back on twice tripped both times.
<!-- Author: Stan Forrester :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->
<!-- Comment: Sat Jul 19 15;42;53 comment by...Stan Forrester -->
when I tried to use the "bars" display to reset the channel the trips occured at 1660 volts and current at 178 (units? Amps?) and again at 1514 volts.  I didn't catch the current in the latter case.  We have tried to reset the channel as many times as the instructions will allow.
<!-- Date: Sat Jul 19 15:37:46.567 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371">166371</A>
 Terminated at 2003.07.19 15:36:14
<!-- Author: RunControl -->


<!-- Date: Sat Jul 19 15:37:47.944 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371">166371</A>
TERMINATE: ene run
<!-- Author: Pierluigi* 2080 -->



<!-- Date: Sat Jul 19 15:41:58.709 2003 -->
The BMU voltage has tripped several times.  We get a trigger 
inhibit.  Experts are paged but we do not hear back. 
 
We are now going to leave channel 84 another off, and start a 
new run.  
  
<!-- Author: Boudreau :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371' target=_top>166371</a>) -->

<!-- Date: Sat Jul 19 15:42:53.414 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166372">166372</A>
TERMINATE: abort cause L3
<!-- Author: Pierluigi* 2080 -->
<!-- Comment: Sat Jul 19 15;51;01 comment by...Pierluigi -->
<pre>L3_RELAY_FAILURE

all nodes seem to be alive in the L3 display. We aborted and reset-partition-config-activate again.</pre>
<!-- Comment: Sat Jul 19 16;02;16 comment by...Pierluigi -->
Sorry, node name 028 is black. (boxes are grey).
<!-- Comment: Sat Jul 19 16;24;20 comment by...Jason Galyardt -->
L3 node 001 also became unreachable some time today (Alberto B. spotted this).
<!-- Comment: Sat Jul 19 18;38;45 comment by...Jason Galyardt -->
Node 001 died late last owl shift when L3 failed to make the End transition after L2 errors forced a run termination/restart.
<!-- Date: Sat Jul 19 15:49:46.242 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166373">166373</A>
 Activated at 2003.07.19 15:48:40
<!-- Author: RunControl -->


<!-- Date: Sat Jul 19 15:49:47.340 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166373">166373</A>
 Activated at 2003.07.19 15:48:40
<!-- Author: RunControl -->


<!-- Date: Sat Jul 19 15:50:45.043 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166373">166373</A>
ACTIVATE: start run with BMU channel 84 off
<!-- Author: Pierluigi* 2080 -->




<!-- Date: Sat Jul 19 15:51:25.655 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8290&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8290' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8291&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8291' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-8292&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-8292' width='300'></a></center>
<!-- Author: Stan Forrester -->


<!-- Date: Sat Jul 19 15:53:01.183 2003 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2003.07.19</td><td>15:44:46</td><td>W Inner BLM</td><td>409.89</td><td>RADS</td></tr> 
<tr><td>2003.07.19</td><td>15:44:46</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.07.19</td><td>15:44:46</td><td>E Inner BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.07.19</td><td>15:44:46</td><td>E Outer BLM</td><td>142.56</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong>
<!-- Author: Stan Forrester :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166373' target=_top>166373</a>) -->


<!-- Date: Sat Jul 19 15:53:06.566 2003 -->
<pre>L2 decision timeout 
 
Error: Startload is still low. ModDone bits 0x4007fdff for event 282281 
TIMEOUT: Mod done not set for slot 12 (0x0007fdff, cnt 3) 
Slot12 saw 3 sequential errors at Jul 19 14:57:41 2003 
FaultManager: Fatal error, sending L2 timeout (L2A 8577)</pre>
<!-- Author: Pierluigi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166373' target=_top>166373</a>) -->
<!-- Comment: Sat Jul 19 16;15;53 comment by...Saltzberg -->
<p>
I think it would be interesting to know if when the silicon
trigger path gets stuck like this, does it happen preferentially
on large events, such as we'd expect for the most
interesting events?
<!-- Date: Sat Jul 19 15:53:51.127 2003 -->
<TABLE bgcolor=#add8e6> 
<TR><TH>Date</th><th>Time</th><th colspan=3>Quantity</th></tr> 
<tr><td>2003.07.19</td><td>15:45:13</td><td>TEVpr</td><td>6459.31</td><td>1E09</td></tr> 
<tr><td>2003.07.19</td><td>15:45:13</td><td>TEVpb</td><td>476.84</td><td>1E09</td></tr> 
<tr><td>2003.07.19</td><td>15:46:49</td><td>C:B0ILUM</td><td>13.52</td><td>E30</td></tr> 
<tr><td>2003.07.19</td><td>15:46:49</td><td>C:B0LLUM</td><td>13.50</td><td>E30</td></tr> 
<tr><td>2003.07.19</td><td>15:46:49</td><td>C:B0PLOS</td><td>2540.42</td><td>Hz</td></tr> 
<tr><td>2003.07.19</td><td>15:46:49</td><td>C:B0ALOS</td><td>34.64</td><td>Hz</td></tr> 
</TABLE> 
<strong>Luminosity and losses</strong>
<!-- Author: Stan Forrester :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166373' target=_top>166373</a>) -->

<!-- Date: Sat Jul 19 15:56:36.457 2003 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166369>
166369</A> 
x289E1</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_05 [5,319,391]</TD>                                       
<TD ALIGN=CENTER>07:22:05</TD>                                          
<TD ALIGN=CENTER>09:17:51</TD>                                          
<TD ALIGN=CENTER>01:44:28</TD>                                          
<TD ALIGN=RIGHT>     83,557,249</TD>                                    
<TD ALIGN=RIGHT>     1,257,248</TD>                                     
<TD ALIGN=RIGHT>     276,374</TD>                                       
<TD ALIGN=RIGHT>  122.218</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166370>
166370</A> 
x289E2</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_05 [5,319,391]</TD>                                       
<TD ALIGN=CENTER>09:23:51</TD>                                          
<TD ALIGN=CENTER>09:31:28</TD>                                          
<TD ALIGN=CENTER>00:00:41</TD>                                          
<TD ALIGN=RIGHT>        155,960</TD>                                    
<TD ALIGN=RIGHT>         4,685</TD>                                     
<TD ALIGN=RIGHT>       1,176</TD>                                       
<TD ALIGN=RIGHT>    0.758</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>0</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166371>
166371</A> 
x289E3</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_05 [5,319,391]</TD>                                       
<TD ALIGN=CENTER>09:38:36</TD>                                          
<TD ALIGN=CENTER>15:36:14</TD>                                          
<TD ALIGN=CENTER>05:35:59</TD>                                          
<TD ALIGN=RIGHT>    218,364,927</TD>                                    
<TD ALIGN=RIGHT>     3,289,943</TD>                                     
<TD ALIGN=RIGHT>     759,402</TD>                                       
<TD ALIGN=RIGHT>  318.420</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>15:55:01</TD>                                                       
<TD ALIGN=CENTER>07:21:10</TD>                                          
<TD ALIGN=RIGHT>     302,078,136</TD>                                   
<TD ALIGN=RIGHT>      4,551,876</TD>                                    
<TD ALIGN=RIGHT>     1,036,952</TD>                                     
<TD ALIGN=RIGHT>  441.396</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->




<!-- Date: Sat Jul 19 16:04:52.092 2003 -->
<font color=red>Shift Summary: </font><pre>We get a stable beam with luminosity of 20E30.  We run stably 
for an 1 1/4 hour until we get L2 timeouts; these take us ~ 1/2 
hour to fix.  Then we run again until the BMU trips at 15:27 
We cannot ramp up the voltages and have not heard from the 
expert so we turn off a single BMU channel and start a new run. 
 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   166369 166371
Delivered Luminosity   468  
Acquired Luminosity    404  
Efficiency             86%
   
</td></tr></table></pre>
<!-- Author: Boudreau -->