<!-- Title: Day shift. Tue May 13, 2003 -->
<!-- SciCo: Kevin McFarland -->
<!-- DAQAce: Ambra Gresele -->
<!-- MonAce: Yi Le -->
<!-- CO: Robert Harr -->
<!-- OpManager: Masa Tanaka -->
<!-- Notes: -->
<pre>store 2540, run 162857 in progress.  Detector/DAQ state OK.  Lum at start of shift 24E30; stack 67E10.</pre>


<!-- Date: Tue May 13 08:04:25.588 2003 -->
<pre>Error (196) >>  (MLE) b0cot17:Messenger:8:03:03 AM->Event 2074946: Bunchcounters 
 in slot 3 (BC=12) and slot 19 (BC=71) disagree (L2B=1)</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 08;24;28 comment by...A.Gresele -->
Error (10) >>  (MLE) b0ccal01:Messenger:8:23:38 AM->Event 2315182: Bunchcounters in slot 16 (BC=118) and slot 17 (BC=115) disagree (L2B=3) 
Error (6) >>  (MLE) b0ccal01:Messenger:8:23:38 AM->Event 2315181: Bunchcounters in slot 16 (BC=53) and slot 17 (BC=118) disagree (L2B=1)


<!-- Comment: Tue May 13 08;53;52 comment by...A.Gresele -->
<pre>(MLE) b0cot19:Messenger:8:52:13 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 6 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:13 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 7 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:13 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 8 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:13 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 9 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:13 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 10 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:13 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 11 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 12 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 13 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 14 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 15 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 16 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 17 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 18 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 19 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Event 2641220: Bunchcounters in slot 5 (BC=65) and slot 20 (BC=77) disagree (L2B=1) 
(MLE) b0cot19:Messenger:8:52:14 AM->Runtime Error 6, Event 2641220: Bunch counter mismatch, mismatch count = 15</pre>
<!-- Comment: Tue May 13 08;57;08 comment by...A.Gresele -->
Error (98) >>  (MLE) b0cot19:Messenger:8:55:18 AM->Event 2673123: Bunchcounters in slot 5 (BC=86) and slot 7 (BC=77) disagree (L2B=2)
<!-- Comment: Tue May 13 09;21;45 comment by...A.Gresele -->
<pre>(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 6 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 7 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 8 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 9 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 10 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 11 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 12 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 13 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 14 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:53 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 15 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:54 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 16 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:54 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 17 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:54 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 18 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:54 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 19 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:54 AM->Event 2943485: Bunchcounters in slot 5 (BC=121) and slot 20 (BC=109) disagree (L2B=3) 
(MLE) b0cot19:Messenger:9:18:54 AM->Runtime Error 8, Event 2943485: Bunch counter mismatch, mismatch count = 15</pre>
<!-- Comment: Tue May 13 09;28;59 comment by...A.Gresele -->
Error (132) >>  (MLE) b0wcal06:Messenger:9:28:05 AM->Event 3048857: Bunchcounters in slot 16 (BC=115) and slot 8 (BC=3) disagree (L2B=2)
<!-- Comment: Tue May 13 09;34;05 comment by...A.Gresele -->
Error (154) >>  (MLE) b0wcal06:Messenger:9:33:27 AM->Event 3104804: Bunchcounters in slot 16 (BC=106) and slot 17 (BC=68) disagree (L2B=0)
<!-- Comment: Tue May 13 09;45;19 comment by...A.Grsele -->
Error (143) >>  (MLE) b0wcal03:Messenger:9:33:06 AM->Event 3104081: Bunchcounters in slot 5 (BC=3) and slot 20 (BC=27) disagree (L2B=1)
<!-- Comment: Tue May 13 10;01;55 comment by...A.Gresele -->
Error (165) >>  (MLE) b0cot16:Messenger:10:01:13 AM->Event 3417271: Bunchcounters in slot 3 (BC=77) and slot 18 (BC=136) disagree (L2B=2)
<!-- Comment: Tue May 13 10;19;06 comment by...A.Gresele -->
Error (175) >>  (MLE) b0cot17:Messenger:10:18:19 AM->Event 3605256: Bunchcounters in slot 3 (BC=77) and slot 15 (BC=109) disagree (L2B=1)
<!-- Comment: Tue May 13 10;52;42 comment by...A.Gresele -->
Error (184) >>  (MLE) b0cot10:Messenger:10:51:45 AM->Event 3964491: Bunchcounters in slot 5 (BC=109) and slot 11 (BC=59) disagree (L2B=2)
<!-- Comment: Tue May 13 11;16;54 comment by...A.Gresele -->
Error (0) >>  (MLE) b0cot10:Messenger:11:12:01 AM->Event 4176046: Bunchcounters in slot 5 (BC=18) and slot 11 (BC=59) disagree (L2B=2)
<!-- Comment: Tue May 13 11;22;57 comment by...A.Gresele -->
Error (26) >>  (MLE) b0wcal06:Messenger:11:22:15 AM->Event 4274517: Bunchcounters in slot 16 (BC=106) and slot 8 (BC=0) disagree (L2B=0)
<!-- Comment: Tue May 13 11;48;32 comment by...A.Gresele -->
Error (38) >>  (MLE) b0cot19:Messenger:11:44:10 AM->Event 4497621: Bunchcounters in slot 5 (BC=18) and slot 7 (BC=112) disagree (L2B=0)
<!-- Comment: Tue May 13 12;07;42 comment by...A.Gresele -->
Error (51) >>  (MLE) b0cot19:Messenger:12:06:35 PM->Event 4708033: Bunchcounters in slot 5 (BC=115) and slot 6 (BC=80) disagree (L2B=3)
<!-- Comment: Tue May 13 12;43;41 comment by...A.Gresele -->
Error (72) >>  (MLE) b0cot18:Messenger:12:43:10 PM->Event 5067922: Bunchcounters in slot 4 (BC=30) and slot 6 (BC=62) disagree (L2B=3)
<!-- Comment: Tue May 13 13;10;01 comment by...A.Gresele -->
Error (96) >>  (MLE) b0wcal05:Messenger:1:07:53 PM->Event 5293132: Bunchcounters in slot 16 (BC=59) and slot 20 (BC=68) disagree (L2B=2)
<!-- Comment: Tue May 13 13;20;35 comment by...A.Gresele -->
Error (109) >>  (MLE) b0cot17:Messenger:1:19:46 PM->Event 5403816: Bunchcounters in slot 3 (BC=124) and slot 19 (BC=80) disagree (L2B=2)
<!-- Comment: Tue May 13 13;41;58 comment by...A.Gresele -->
Error (119) >>  (MLE) b0cot10:Messenger:1:41:04 PM->Event 5601334: Bunchcounters in slot 5 (BC=77) and slot 11 (BC=115) disagree (L2B=0)
<!-- Comment: Tue May 13 14;02;46 comment by...A.Gresele -->
Error (128) >>  (MLE) b0cot01:Messenger:2:01:30 PM->Event 5787704: Bunchcounters in slot 2 (BC=6) and slot 14 (BC=3) disagree (L2B=0)
<!-- Comment: Tue May 13 14;32;22 comment by...A.Gresele -->
Error (128) >>  (MLE) b0cot01:Messenger:2:01:30 PM->Event 5787704: Bunchcounters in slot 2 (BC=6) and slot 14 (BC=3) disagree (L2B=0)
<!-- Comment: Tue May 13 14;52;54 comment by...A.Gresele -->
Error (145) >>  (MLE) b0cot01:Messenger:2:52:16 PM->Event 6237491: Bunchcounters in slot 2 (BC=74) and slot 13 (BC=106) disagree (L2B=0)
<!-- Comment: Tue May 13 15;20;55 comment by...A.Gresele -->
Error (159) >>  (MLE) b0cot15:Messenger:3:20:01 PM->Event 6477371: Bunchcounters in slot 3 (BC=109) and slot 12 (BC=27) disagree (L2B=2)
<!-- Comment: Tue May 13 15;52;38 comment by...A.Gresele -->
b0dap73.fnal.gov:Thread-18810:3:49:14 PM->Done Timeout: COT_19 
<!-- Comment: Tue May 13 16;03;22 comment by...Ambra -->
(MLE) b0wcal03:Messenger:4:00:14 PM->Event 6815701: Bunchcounters in slot 5 (BC=9) and slot 20 (BC=24) disagree (L2B=3)
<!-- Date: Tue May 13 08:32:07.003 2003 -->
<pre>Error (61) >>   SCPU b0eb15.  
Error (47) >>   SCPU b0eb15 (forwarded by FER crate CCAL_09).  
Error (38) >>  SCPU-P0-E-TracerEventId: Event 2398981, crate 10, channel 9 has e 
ither bad Tracer ID or bad markers around Tracer word. 
Error (31) >>   SCPU b0eb15 (forwarded by FER crate CCAL_11).  
Error (22) >>  SCPU-P0-E-TracerEventId: Event 2398981, crate 12, channel 8 has e 
ither bad Tracer ID or bad markers around Tracer word.</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->


<!-- Date: Tue May 13 08:37:36.098 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-4927&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-4927  width='300' align='top'></td><td>Some TrigMon plots are showing more errors than normal.  First, the Clist word validation plot:</a></td></tr></table>
<!-- Author: Robert Harr -->


<!-- Date: Tue May 13 08:39:10.909 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-4928&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-4928  width='300' align='top'></td><td>Second, the L1 Cal DIRAC Occupancy plot.</a></td></tr></table>
<!-- Author: Robert Harr -->


<!-- Date: Tue May 13 08:41:46.249 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-4929&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-4929  width='300' align='top'></td><td>And third, the XTRP Summary canvas.</a></td></tr></table>
<!-- Author: Robert Harr -->




<!-- Date: Tue May 13 09:00:29.925 2003 -->
The errors documented above are higher than typical during the last few days.  It's not clear that they are a problem yet -- the references indicate that error rates in this range (few per thousand events) are acceptable.  I'll check again in two hours and, if the errors double, contact the experts.	
<!-- Author: Robert Harr :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 10;54;41 comment by...Robert Harr -->
It's two hours later and the only real change is in the XTRP plot.  The number of errors is up to 14 in 2060 events.  That's 7 per 1000 events and the reference says that 1 per 1000 is OK.  I wrote to Nathan Eddy and Jim Kraus earlier about this.  We've now paged the XTRP expert.
<!-- Comment: Tue May 13 11;14;42 comment by...Robert Harr -->
Jim Kraus came by and explained that 1 error per bin per 1000 events is an acceptable level.  The above number is integrated over the entire plot, and the error per bin rates are within the allowed levels.
<!-- Date: Tue May 13 09:13:38.185 2003 -->
<pre> 
10'42" b0dap57.fnal.gov:resmgr.ReplyMonitor 4:9:11:03 AM-> B0PLOS = 6916.8583984375 B0ALOS = 207.8522186279297 Trigger Rate Excursion : PreFred trigger bit 3 </pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->


<!-- Date: Tue May 13 09:15:54.752 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-4930&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-4930  width='300' align='top'></td><td>RATE history plot for bit 3</a></td></tr></table>
<!-- Author: A.Gresele -->


<!-- Date: Tue May 13 09:17:21.919 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4931&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4931' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4932&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4932' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4933&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4933' width='300'></a></center>
<!-- Author: Yi Le -->
<!-- Comment: Tue May 13 09;18;59 comment by...Yi Le -->
acnet hourly plots


<!-- Date: Tue May 13 09:20:18.769 2003 -->
<pre>Error (108) >>  (MLE) b0dap73.fnal.gov:Thread-18810:9:17:52 AM->TriggerInhibitStatus:  SVX</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 13;19;31 comment by...A.Gresele -->
<pre>another.</pre>
<!-- Comment: Tue May 13 15;17;21 comment by...A.Gresele -->
... again.
<!-- Date: Tue May 13 10:03:53.128 2003 -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>Concerning the recent "b0tsi02" errors: 
======================================= 
 
When you see an error such as  
TS L2 FSM failed to start (State 14) 
this usually implies that a front-end  
crate is asserting ERROR or failing to  
properly assert the DONE or BUSY lines. 
 
When you see this TS error message you need to  
look in the Error Logger to see which crate is  
actually causing the problem 
 
Example 1: Run 162744 
 
(MLE) b0tsi00:Messenger:10:05:50 AM->TS L2 FSM failed to start (State 14) 
5'40" 1 crate/s: b0l1cl00(128),  in error.[RXPT] 
 
The CLC crate was asserting CDF_ERROR. 
 
Example 2: Runs 162817, 162818, 162819 (Sunday morning) 
 
the ghost tracer in b0svx06 was seen to be asserting BUSY. 
This was actually a DB problem fixed by  restarting runControl. 
</pre></td></tr></td></table>
<!-- Author: Greg Feild -->
<!-- Comment: Tue May 13 13;22;54 comment by...Andy H. -->
b0l1cl00 is a CLC crate?
<!-- Comment: Wed May 14 08;26;00 comment by...Greg -->
<pre>No, sorry. That is a Level 1 calorimeter crate.
</pre>
<!-- Date: Tue May 13 10:12:40.896 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4934&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4934' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4935&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4935' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4936&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4936' width='300'></a></center>
<!-- Author: Yi Le -->
<!-- Comment: Tue May 13 10;13;42 comment by...Yi Le -->
acnet hourly plots



<!-- Date: Tue May 13 10:50:05.042 2003 -->
<pre>From the beginning of this shift we find a lot of TEV Mon warning  
due to the spikes in the PLOST. In fact: 
-> 8:08:34 TEV Mon Warning! 
-> 8:13:56 again 
-> 8:14:00 again 
-> 8:14:10 again 
-> 8:23:59 again 
-> 8:24:24 again 
-> 10:38:32 again</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->




<!-- Date: Tue May 13 11:06:08.442 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4937&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4937' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4938&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4938' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4939&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4939' width='300'></a></center>
<!-- Author: Yi Le -->
<!-- Comment: Tue May 13 11;06;37 comment by...Yi Le -->
acnet hourly plots




<!-- Date: Tue May 13 11:20:08.977 2003 -->
<pre>b0tsi00 in VxWorks has "T" red. In fact we find: 
t46        ROBIN_Object   b52288 200 SUSPEND      233394   b51d68   d0003     0 
t48        ROBIN_Object   b43950 200 SUSPEND      233394   b43430   d0003     0 
t50        ROBIN_Object   b36440 200 SUSPEND      233394   b35f20   d0003     0 
t52        ROBIN_Object   adf3b8 200 SUSPEND      233394   adee98   d0003     0 
t54        ROBIN_Object   acaa78 200 SUSPEND      233394   aca558   d0003     0 
</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 11;25;31 comment by...A.Gresele -->
<pre>Now it is green!</pre>
<!-- Comment: Tue May 13 11;32;17 comment by...SCN -->
These were "Mostly Harmless" suspended ROBIN tasks.  They didn't
affect data taking but only turned the VxWorks button red.  I deleted them and the button turned green.  Nice spot by
the Aces.
<!-- Date: Tue May 13 11:21:31.409 2003 -->
<pre>Error (21) >>  (MLE) b0l2de00:Messenger:11:18:51 AM->CListMonitor: Sync error in L2 pipeline, event 4241447</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->



<!-- Date: Tue May 13 11:24:02.840 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-4940&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-4940  width='300' align='top'></td><td>CEM tower West 6 - 7R is a bit warm today.</a></td></tr></table>
<!-- Author: Robert Harr -->
<!-- Comment: Tue May 13 13;30;49 comment by...Robert Harr -->
This channel continues to worsen.  Occupancy is now about 0.3.
<!-- Comment: Tue May 13 15;50;04 comment by...m mattson, v tiwari -->
<br>
Robert also looked at the higher threshold (200 ADC) plot, where this channel does *not* appear hot.  So it is unlikely to affect trigger rates.<p>
Stopped by ArgoBob's office, where he mentioned he saw your e-log entry.  He and Larry will discuss what needs to be done during the next (supervised) access.
<!-- Comment: Tue May 13 16;53;46 comment by...Larry -->
As long as the high threshold plot shows nothing, we only ask 
for access if there is some opportunity for other reasons, 
for Bob to go up and investigate. (low priority)
<!-- Date: Tue May 13 11:33:20.249 2003 -->
<pre>SVX pink cell B5W11L4 AVDD, unmark it, turns green now</pre>
<!-- Author: Yi Le -->



<!-- Date: Tue May 13 11:55:03.285 2003 -->
<pre>Problem with L2 and L1 in the XMon. We check the clc00 status: 
288aec vxTaskEntry    +60 : 0 () 
value = 0 = 0x0 
</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 12;00;40 comment by...A.Gresele -->
<pre>... In fact b0clc00 in VxWorks has "T" red.</pre>

<!-- Comment: Tue May 13 12;01;19 comment by...A.Gresele -->
<pre>receive    1              e81128 200 SUSPEND           0   e810d0       0     0
</pre>

<!-- Comment: Tue May 13 12;06;48 comment by...A.Gresele -->
<pre>shephered it manually and now it is working</pre>



<!-- Date: Tue May 13 12:10:31.309 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4941&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4941' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4942&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4942' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4943&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4943' width='300'></a></center>
<!-- Author: Yi Le -->


<!-- Date: Tue May 13 12:17:28.737 2003 -->
<pre>silicon pink cell, happend about 12:00 
SVX B1W8L3 DVDD 
L00 B1W3L3,B1W0L3,B1W5L3,B1W5L0 all DVDD 
unmarked them,turn green now</pre>
<!-- Author: Yi Le -->


<!-- Date: Tue May 13 12:37:24.752 2003 -->
<pre>Proc Mon error in FCDFSGI1 Tape.</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 15;11;09 comment by...A.Gresele -->
<pre>Now it's ok!</pre>
<!-- Comment: Tue May 13 15;23;44 comment by...A.Gresele -->
<pre>... again red!!!</pre>
<!-- Date: Tue May 13 12:37:48.723 2003 -->
Nearly all the L1 and L2 trigger rates went cold (blue) in XMon.  After discussing the problem, we decided to kill and restart the XMon consumer.  Now the trigger summary is good.
<!-- Author: Robert Harr :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->



<!-- Date: Tue May 13 12:54:26.409 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4944&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4944' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4945&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4945' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4946&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4946' width='300'></a></center>
<!-- Author: Yi Le -->


<!-- Date: Tue May 13 12:58:47.465 2003 -->
<pre>b0pcal07 has "H" red in VxWorks. In fact: 
Mon_III    FER_monitorI  1a5a690 220 DELAY        280cdc  1a5a5f0  1c0001    22 
 
and we have this error : 
0x1ac6dd0 (Messenger): Heap Error: 'Internal Inconsistency' at  199ac48 (count = 28077)</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 13;06;49 comment by...m mattson -->
<br>VxWorks instructions said to page the Cal ADMEM pager, so they did.  I'm fine with Ambra shepherding crate b0pcal07.


<!-- Date: Tue May 13 13:09:33.897 2003 -->
<pre>b0pcal07 was red in VxWorks. 
shepherd it manually</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->







<!-- Date: Tue May 13 13:36:42.090 2003 -->
MCR <a 
href=http://www-bd.fnal.gov/cgi-mcr/elog.pl?nb=2003&action=view&page=360&anchor=133323&hilite=13:33:23- 
target=_top>plan</a> is to drop store between 1900-2000.  
<!-- Author: Kevin :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->



<!-- Date: Tue May 13 13:53:07.638 2003 -->
Boaz Klima from D0 calls.  They are putting in the Roman Pots.
<!-- Author: Kevin  :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->




<!-- Date: Tue May 13 13:59:32.302 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4950&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4950' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4951&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4951' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4952&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4952' width='300'></a></center>
<!-- Author: Yi Le -->
<!-- Comment: Tue May 13 14;04;06 comment by...Yi Le -->
<pre>spiky in B0 loss plot</pre>




<!-- Date: Tue May 13 15:02:52.907 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4953&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4953' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4954&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4954' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-4955&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-4955' width='300'></a></center>
<!-- Author: Yi Le -->


<!-- Date: Tue May 13 15:08:45.181 2003 -->
<pre>SlowControl Icicle heart beat not updated since 2003-05-13 14:55:45</pre>
<!-- Author: A.Gresele :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->
<!-- Comment: Tue May 13 15;14;57 comment by...A.Gresele -->
<pre>...again!</pre>






<!-- Comment: Tue May 13 16;10;05 comment by...Steve Hahn -->
<p>According to Ambra, she had to restart ICICLE twice.  There are "ICICLE missing" errors in the SounderServer at 1504 and 1514.  However, there has not been an error since.  When I looked at the SOLENOID2 display, I found there were <b>two</b> Visual C++ processes running, I killed both.  I'm not sure if this could have caused problems, but it does not seem like a healthy thing to do.</p>
<!-- Date: Tue May 13 15:55:16.356 2003 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857>
162857</A> 
x27C29</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_04 [8,278,368]</TD>                                       
<TD ALIGN=CENTER>05:19:31</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>10:12:07</TD>                                          
<TD ALIGN=RIGHT>    509,280,384</TD>                                    
<TD ALIGN=RIGHT>     6,754,439</TD>                                     
<TD ALIGN=RIGHT>   2,062,005</TD>                                       
<TD ALIGN=RIGHT>  802.492</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>15:55:02</TD>                                                       
<TD ALIGN=CENTER>10:12:07</TD>                                          
<TD ALIGN=RIGHT>     509,280,384</TD>                                   
<TD ALIGN=RIGHT>      6,754,439</TD>                                    
<TD ALIGN=RIGHT>     2,062,005</TD>                                     
<TD ALIGN=RIGHT>  802.492</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Tue May 13 16:00:31.941 2003 -->
<font color=red>Shift Summary: </font><pre>Highlights 
---------- 
0800 Inherit store 2540, run 162857 in progress.   
     B0 Lumi at start of shift 24E30 
0815 Spiky losses; TeVMon warnings.  No action taken. 
1040 Several spiky losses within 15 minutes.  Steve Nahn 
     leaves us instructions to contact MCR if 
     we get three additional TeVMon warnings within an hour. 
1100 Investigate XTRP cal mismatches in TrigSim over spec. 
     Jim Kraus indicates that we have misunderstood the spec 
     and that error rate observed is OK. 
1133 SVX B5W11L4 AVDD pinkie.  Transient 
1155 XMon rates indicate that CLC_RECEIVE has died again.  Restart it 
     and then XMon. 
1310 b0pcal07 goes red in VXWorks monitor (memory corruption).  Shepherd 
     and recover. 
1505 Lose the ICICLE connection.  Steve Hahn investigating. 
 
Stores 
------ 
2540 in progress at start of shift 
 
Runs 
---- 
162857: in progress at start of shift (started 0519) 
        in progress at end of shift.  La plus ca change... 
 
Problems 
-------- 
* CEM W67R is moderately hot 
* ICICLE still down at end of shift 
 
       delivered   572.6 nb-1 
       acquired    552.8 nb-1 
       efficiency  96.5</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   162857
Delivered Luminosity   574.6  
Acquired Luminosity    555.8  
Efficiency             96.6%
   
</td></tr></table></pre>
<!-- Author: Kevin -->


<!-- Date: Tue May 13 16:03:20.674 2003 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>W Inner BLM</td><td>543.47</td><td>RADS</td></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>E Inner BLM</td><td>2.71</td><td>RADS</td></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>E Outer BLM</td><td>405.06</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong>
<!-- Author: Yi Le -->



<!-- Date: Tue May 13 16:03:42.120 2003 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>W Inner BLM</td><td>543.47</td><td>RADS</td></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>E Inner BLM</td><td>2.71</td><td>RADS</td></tr> 
<tr><td>2003.05.13</td><td>16:02:12</td><td>E Outer BLM</td><td>405.06</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong> 
  
<!-- Author: Yi Le -->


<!-- Date: Tue May 13 16:07:16.923 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-4962&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-4962  width='300' align='top'></td><td>shift lumi.</a></td></tr></table>
<!-- Author: Yi Le -->



<!-- Date: Tue May 13 16:10:30.810 2003 -->
Three times ProcMon gave an error due to fcdfsgi1 tape.
<!-- Author: Jonatan :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=162857' target=_top>162857</a>) -->