<!-- Title: Eve shift. Tue Feb  8, 2005 -->
<!-- SciCo: Xin Wu -->
<!-- DAQAce: Geum Bong Yu -->
<!-- MonAce: Cristobal Cuenca -->
<!-- CO: James Belliger -->
<!-- OpManager: Bob Wagner -->
<!-- Notes: -->
<pre>Inherited store 3972, Lum = 43E30. Pb atack 40E10. Stacking rate 0. 
Store will be kept until probably tomorrow morning. 
 
Plan: take data until 2.5 hour before end of store then Pulsar test.</pre>


<!-- Date: Tue Feb 8 16:24:51.124 2005 -->
<br> <b> CMX HV trip: </b> miniskirt 16-17 <br> 
HV reset once <br>
<!-- Author: Cristobal :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464' target=_top>193464</a>) -->


<!-- Date: Tue Feb 8 16:26:16.388 2005 -->
<pre> 16:02:03 
 
 Attention!!!. SCPU_TRACER_EVENT_ID Error !!!  
 Hardware EVB has detected a problem with data quality in  
 SCPU b0eb15 (forwarded by FER crate CCAL_00).  
 
(MLE) b0svx06:Messenger:4:02:12 PM->Silicon Timeout:BUSY- Slots:  09:fa00 10:fa20 12:fa40 16:f800 18:f820 20:f840 
 
(MLE) b0dap73.fnal.gov:Thread-4861:4:02:43 PM->  Level 2 Decision Timeout 
 
(MLE) b0dap73.fnal.gov:Thread-4861:4:03:28 PM->  Level 2 Decision Timeout 
 
(TS)   16:03:33 ********* L2 trigger rate has dropped to 0 Hz! Events are not being logged ! 
 
6'27" b0l3pcom1.fnal.gov:main:4:06:27 PM->Host b0eb11.fnal.gov, task tRec_0 
SCPU-P1-E-TracerEventId: Event 510891, crate 2, channel 1 has either bad Tracer ID or bad markers around Tracer word. 
 Tracer word event ID is 0, should be 11. 
 
(MLE) b0pcal09:Messenger:4:12:46 PM->Runtime Error 1, Event 611516: See nonsequential event number (old 11, new 10) in L2 message 
(MLE) b0l3pcom1.fnal.gov:main:4:12:46 PM->Host b0eb16.fnal.gov, task tRec_0 
SCPU-P1-E-VrbHeader: Dump of header words for event 611516 from VRB in slot 12: 
0x00000000 0x00004370 0x00000000 0x11ac08bc 0x08bc0dd8 0x073007f4 0x031c0000 0x00000000 
(MLE) b0dap73.fnal.gov:Thread-4861:4:12:50 PM->Done Timeout: PCAL_09 
CT: 2005.02.08 16:12:54 
12'54" 1 crate/s: b0pcal09(43),  Done TO. [RXPT] 
 
 These errors occured in series. Sofia, previous shift also had had these errors every 10 minuites.  </pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464' target=_top>193464</a>) -->


<!-- Date: Tue Feb 8 16:40:15.727 2005 -->
As a request of evb expert, checked SM+SCPU, every SCPU is ok now. The error-orange window- message saied it could be a evb problem, but it's not a problem of evb but cot readout.   
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464' target=_top>193464</a>) -->


<!-- Date: Tue Feb 8 16:43:45.897 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2215&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2215' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2216&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2216' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2217&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2217' width='300'></a></center>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 16;44;26 comment by...Cristobal -->
<pre>Hourly plots</pre>


<!-- Date: Tue Feb 8 16:56:17.924 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2218&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2218  width='300' align='top'></a></td><td>We see a spike in B0PAGC when D0 inserts their Roman Pots.</td></tr></table>
<!-- Author: Bob Wagner (ANL) -->
<!-- Comment: Tue Feb  8 16;57;27 comment by...Bob Wagner (ANL) -->
MCR Log says D0 pots went in at 16:36.



<!-- Date: Tue Feb 8 16:59:24.868 2005 -->
<pre>47'26" b0l3pcom1.fnal.gov:main:4:47:26 PM->Host b0eb11.fnal.gov, task tRec_0 
SCPU-P1-E-TracerEventId: Event 1085997, crate 52, channel 3 has either bad Tracer ID or bad markers around Tracer word. 
 Tracer word event ID is 13, should be 13. 
 
 Hardware EVB has detected a problem with data quality in  
 SCPU b0eb11 (forwarded by FER crate COT_03).  
 
****************************************************** 
 
47'26" b0l3pcom1.fnal.gov:main:4:47:26 PM->Host b0eb11.fnal.gov, task tRec_0 
SCPU-P1-E-TracerEventId: Event 1085997, crate 0, channel 4 has either bad Tracer ID or bad markers around Tracer word. 
 Tracer word event ID is 0, should be 13. 
 
 Hardware EVB has detected a problem with data quality in  
 SCPU b0eb11 ( FER crate : NOT_AVAILABLE).  
 
******************************************* 
 
CT: 2005.02.08 16:51:05 
51'5" 1 crate/s: b0svx06(16),  busy. [RXPT] 
(MLE) b0dap73.fnal.gov:Thread-4861:4:51:05 PM->Busy Timeout: VRB_ISL_06 </pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464' target=_top>193464</a>) -->

<!-- Date: Tue Feb 8 17:01:33.585 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464">193464</a>
 Terminated at 2005.02.08 16:59:57
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 17:01:34.369 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464">193464</a>
TERMINATE: too many done TO with several crates. end this run
<!-- Author: Geumbong (2080) -->


<!-- Date: Tue Feb 8 17:11:32.895 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193465">193465</a>
 Activated at 2005.02.08 17:11:23
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 17:11:40.294 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193465">193465</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,535] 
<!-- Author: Geumbong (2080) -->


<!-- Date: Tue Feb 8 17:15:17.869 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193465">193465</a>
 Terminated at 2005.02.08 17:14:53
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 17:16:23.075 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193465">193465</a>
TERMINATE: didn't go ACTIVATE state. SVX crates and b0tsi00 were hung up. try again. 
<!-- Author: Geumbong (2080) -->



<!-- Date: Tue Feb 8 17:18:35.862 2005 -->
after got L2TO, HRR issed. But in the transition to recorvering, RC was stuck with red b0tsi00 crates. Frank, was near RC, tried shepherd the crates, but failed with doneTOs of many cot+etc..crates. we had to end the run. During ending that run, we got a message to "do clean up evb". So we do it and check SCPUs.  
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464' target=_top>193464</a>) -->
<!-- Comment: Tue Feb  8 18;03;36 comment by...Geumbong -->
Now we take partition 0 with main daq RC.
<!-- Date: Tue Feb 8 17:25:32.616 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2219&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2219' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2220&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2220' width='300'></a></center>
<!-- Author: Cristobal -->


<!-- Date: Tue Feb 8 17:38:47.501 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2221&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2221' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2222&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2222' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2223&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2223' width='300'></a></center>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 18;11;03 comment by...<b>Cristobal </b> -->
Hourly plots: <b>16:00 to 17:30</b>
<!-- Date: Tue Feb 8 17:41:03.213 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193466">193466</a>
 Activated at 2005.02.08 17:40:55
<!-- Author: RunControl -->



<!-- Date: Tue Feb 8 17:41:04.418 2005 -->
<pre>L3proxy lost connection with rt server.  
Sympthoms: Commands from RC do not get through. 
Restarted Level3 Proxy. Seems fine now.   </pre>
<!-- Author: Boris.  -->

<!-- Date: Tue Feb 8 17:41:53.377 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193466">193466</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,535] 
<!-- Author: Geumbong (2080) -->



<!-- Date: Tue Feb 8 17:42:38.764 2005 -->
<b> CPR trips: </b> CPR complained already twice. <br>When opening the CPR window, all the sections were green. However the summary bars (Voltage, Current and Average Current) show values out of order: <font color=red><b> 999900.1% </b> and <b> -999900.1% </b></font><br>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 17;44;02 comment by...Bob Wagner (ANL) -->
I believe this display relates to the old gas CPR and is out of date.


<!-- Comment: Wed Feb  9 09;47;48 comment by...Phil Schlabach -->
those funky values are coming from the muon pc, cpr types need to change where this comes from (i.e. the tags displayed are wrong)
<!-- Date: Tue Feb 8 17:47:01.890 2005 -->
<pre><b>TE-CHLR_1</b>: went to <b>-2.2 C</b> </pre>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 18;03;50 comment by...Cristobal -->
Expert paged... and coming <br>
Meanwhile, cryo tech got the value stable at <b>-1.7</b>. <br>
Silicon expert: <b>proceed</b> with data taking! <br>


<!-- Date: Tue Feb 8 18:05:19.503 2005 -->
<br>There is a pink cell in silicon: <b>B5 W6 L2</b>.<br> 
Expert is having a look <br>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 18;08;16 comment by...Cristobal -->
<br>Expert required to <b>HRR</b><br> and it worked!<br>



<!-- Date: Tue Feb 8 18:17:04.001 2005 -->
<pre> Attention!!!. SCPU_TRACER_EVENT_ID Error !!! 
 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb16 (forwarded by FER crate CCAL_07). 
 Datataking is paused: events are not being processed. 
 
^^ HRR helped it. </pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193466' target=_top>193466</a>) -->

<!-- Date: Tue Feb 8 18:20:33.755 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193466">193466</a>
 Terminated at 2005.02.08 18:20:24
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 18:20:50.196 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193466">193466</a>
TERMINATE: end run
<!-- Author: Geumbong (2080) -->
<!-- Comment: Tue Feb  8 18;29;00 comment by...Geumbong -->
<pre>We got same error with evb

 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb11 ( FER crate : NOT_AVAILABLE). 

and followed by 

(MLE) b0dap73.fnal.gov:Thread-6113:6:19:05 PM->Busy Timeout: VRB_ISL_06 

tried HRR. but RC was stuck in halted state with yellow b0tsi02 crate. We had to abort that run </pre>
<!-- Date: Tue Feb 8 18:24:31.458 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193467">193467</a>
 Activated at 2005.02.08 18:23:55
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 18:24:32.670 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193467">193467</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,535] 
<!-- Author: Geumbong (2080) -->



<!-- Date: Tue Feb 8 18:42:22.003 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193467">193467</a>
 Terminated at 2005.02.08 18:41:24
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 18:42:23.396 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193467">193467</a>
TERMINATE: end this run with same error as previous run. 
<!-- Author: Geumbong (2080) -->


<!-- Date: Tue Feb 8 18:53:02.604 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468">193468</a>
 Activated at 2005.02.08 18:52:05
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 18:53:03.800 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468">193468</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,535] 
<!-- Author: Geumbong (2080) -->



<!-- Date: Tue Feb 8 18:59:35.519 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2224&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2224' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2225&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2225' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2226&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2226' width='300'></a></center>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 19;01;12 comment by...Cristobal -->
<font color=red>Hourly plots: <b>17:00</b> to <b>18:00</b></font>
<!-- Date: Tue Feb 8 19:01:04.531 2005 -->
First time, we got voice alarm that "Ack net may not be sending updates", so checked icicle log which said "lost connection to the RT server". We checked RTserver and found many crates were closed.  At that time, auto HRR issued because of busy TO "Silicon Timeout:BUSY- Slots:  20:e1a0". During HRR, RC was stuck with same symptom with previous run, red b0wvx06 and yellow b0tsi02 crates. After ending run, we check rtserver again and every crates were established. Now everything is ok. we start new run, 193468. 
<!-- Author: Geumbong -->
<!-- Comment: Tue Feb  8 20;17;02 comment by...Geumbong -->
I want to correct this comment. It was not "silicon busy TO" but "data quality problem in SCPU b0eb20" to induce HRR. 
<!-- Date: Tue Feb 8 19:06:25.509 2005 -->
<pre>
CT: 2005.02.08 18:56:27 
56'27" 1 crate/s: b0pcal04(36),  Done TO. [RXPT] 
 
Tried HRR, but it happened again. shepherded this crate. We also got the error also. 
 
 Attention!!!. SCPU_TRACER_EVENT_ID Error !!! 
 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb16 (forwarded by FER crate COT_16). 
 
HRR issued. it's ok for now. 
</pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468' target=_top>193468</a>) -->


<!-- Date: Tue Feb 8 19:08:14.521 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2227&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2227' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2228&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2228' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2229&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2229' width='300'></a></center>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 19;10;02 comment by...Cristobal -->
<font color=cyan> Hourly plots: <b>18:00</b> to <b>19:00</b></font>


<!-- Date: Tue Feb 8 19:14:20.064 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2230&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2230  width='300' align='top'></a></td><td>Pinky  unmarked</td></tr></table>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 19;59;11 comment by...Cristobal -->
<b><font color=pink>B5W6L2</font></b>
<!-- Date: Tue Feb 8 19:16:30.496 2005 -->
<font color=green><b>CPR HV</b></font> complained again!
<!-- Author: Cristobal :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468' target=_top>193468</a>) -->
<!-- Comment: Tue Feb  8 19;17;51 comment by...Cristobal -->
No action taken as it went to <font color=green>GREEN</font> immediately


<!-- Date: Tue Feb 8 19:25:53.728 2005 -->
<pre>Got an error message that "TevMon failing to receive Acnet messages wia RTserver". Icicle worked with saying "slow control" and every crate was established(Does it mean nothing?) in rtserver. I also checked following. 
*********************************************************** 
b0dap33:275:~>   ps -lwu cdfdaq | grep rtserver 
000 S  5430 31776     1  7  75   0    - 41033 schedu ?        01:12:28 rtserver.x 
000 S  5430  4058     1  0  76   0    -   881 rt_sig ?        00:00:03 rtserver_check. 
*********************************************************** 
Now it's just ok. Tevmon is also updating. we haven't got that error any more, but.. I'm wondering what can I do for it if things happen again?</pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468' target=_top>193468</a>) -->
<!-- Comment: Wed Feb  9 08;38;39 comment by...Ray Culbertson -->
The data comes from the accelerator to the AcnetMonitor
running on b0dap85, from there to rtserver, from there to TevMon.
TevMon should get an update every 2s, it complains if it has been 15s.  If it complains, first wait a few minutes, it is usually just  a slow-down that recovers itself.  (might want to warn mon ace and scico that we could be unprotected for a minute - watch acnet directly).  If it does not start updating, try to restart acnetMonitor (see important processes page).  If necessary, then restart rtserver,then page DAQ.  Note that the rtserver has several failure modes.  A complete failure will cause VXMon to go red, L3 to go gold and RC commands will fail. However sometimes just one connection get stuck or dropped, so it is reasonable to try to restart it in this case.

<!-- Date: Tue Feb 8 19:29:31.596 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468">193468</a>
 Terminated at 2005.02.08 19:29:23
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 19:29:44.858 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468">193468</a>
TERMINATE: RC is frozen
<!-- Author: Geumbong (2080) -->
<!-- Comment: Tue Feb  8 19;50;39 comment by...Geumbong -->
During HRR because of "data quality in SCPU b0eb14", RC was frozen with two yellow crates, b0fib02 and b0wcal01. We aborted tun. 
<!-- Comment: Tue Feb  8 19;53;56 comment by...Geumbong -->
<pre>
At same time, we got daqerrmon error. Daqerrmon was restarted.

CT: 2005.02.08 19:25:56
25'56" b0dap84.fnal.gov:ConsumerErrorRe:7:25:56 PM->DAQErrMon:DataMissingError: Error 1 Event 0 Run 193468 Bank CLTD_StorableBank missing from event record, nErrors 1  

 Attention !!!.  CER_DAQERRMON_DATA_MISSING_ERROR !!! 
 
 DAQErrMon:DataMissingError: 
 Error 1 Event 0 Run 193468 Bank CLTD_StorableBank missing from event record, nErrors 1 
 HRR is STRONGLY RECOMMENDED. 
</pre>
<!-- Date: Tue Feb 8 19:33:12.371 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469">193469</a>
 Activated at 2005.02.08 19:33:05
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 19:33:28.550 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469">193469</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,535] 
<!-- Author: Geumbong (2080) -->



<!-- Date: Tue Feb 8 19:46:22.647 2005 -->
<pre>Got this error, HRR issued.  
 
 Hardware EVB has detected a problem with data quality in  
 SCPU b0eb15 ( FER crate : NOT_AVAILABLE).  
 
After that, we got daqerrmon error.  
 
CT: 2005.02.08 19:39:58 
39'58" b0dap84.fnal.gov:ConsumerErrorRe:7:39:58 PM->DAQErrMon:DataMissingError: Error 1 Event 0 Run 193469 Bank EMTD_StorableBank missing from event record, nErrors 1  -->   
 Additional Information:  
 
 Attention !!!.  CER_DAQERRMON_DATA_MISSING_ERROR !!!  
  
 DAQErrMon:DataMissingError:  
 Error 1 Event 0 Run 193469 Bank EMTD_StorableBank missing from event record, nErrors 1  
 HRR is STRONGLY RECOMMENDED.</pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469' target=_top>193469</a>) -->




<!-- Date: Tue Feb 8 19:56:58.744 2005 -->
From the Run Coordinator E-log: 
 
During turn-on after the access, the Main Injector MI-50 QD cubicle sustained damage when the Vacuum Circuit Breaker was closed in. Due to the seriousness of the failure and the need to fully understand the cause of the damage, the Main Injector will be left off tonight. It may be off for a considerably longer time, the course of action will be discussed tomorrow. 
<!-- Author: Xin -->


<!-- Date: Tue Feb 8 19:57:46.416 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2231&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2231  width='300' align='top'></a></td><td>Unmarked pinky (again) but now with a pretty rough peak</td></tr></table>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 20;00;13 comment by...Cristobal -->
<b><font color=pink>B5W6L2</font></b>


<!-- Date: Tue Feb 8 20:02:05.702 2005 -->
DAQErrMon was logging no events, and it seemed to have a severe error back when trying to connect to a socket.  This being the second attempt to get DAQErrMon running, I killed all consumers to start fresh.  Got the same error again, but this time it seems to work and we've no voice messages complaining that it wasn't there.<BR> 
%ERLOG-s ROOT/TUnixSystem: bind (Address already in use)  GlobalLibraryLogger<BR> 
      TUnixSystem::UnixTcpService() 08-Feb-2005 19:51:28 CST<BR> 
         run = 0 event = 0<BR> 
DisplayServer: Connected successfully to CONSUMER socket at port 9051 on host b0dap81.fnal.gov.  
<!-- Author: James :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469' target=_top>193469</a>) -->


<!-- Date: Tue Feb 8 20:06:34.346 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2232&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2232' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2233&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2233' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2234&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2234' width='300'></a></center>
<!-- Author: Cristobal; -->
<!-- Comment: Tue Feb  8 20;07;53 comment by...Cristobal -->
Hourly plots: <font color=yellow><b>19:00</b> to <b>20:00</b></font>



<!-- Date: Tue Feb 8 20:22:58.367 2005 -->
<pre> Attention!!!. SCPU_BAD_CHANNEL_COUNTS Error !!! 
 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb14. 
 
In addition to this error, we also got daqerrmon error.  
 
 Attention !!!.  CER_DAQERRMON_DATA_MISSING_ERROR !!! 
 
 DAQErrMon:DataMissingError: 
 Error 1 Event 0 Run 193469 Bank CLTD_StorableBank missing from event record, nErrors 1 
 HRR is STRONGLY RECOMMENDED. 
 
After HRR issued, they are gone.  </pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469' target=_top>193469</a>) -->


<!-- Date: Tue Feb 8 20:28:25.120 2005 -->
<pre> Attention!!!. SCPU_BAD_VRB_BYTE_COUNT Error !!! 
 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb14. 
  
followed by  
 
 Attention !!!.  CER_DAQERRMON_DATA_MISSING_ERROR !!! 
 
 DAQErrMon:DataMissingError: 
 Error 5 Event 0 Run 193469 Bank CLTD_StorableBank missing from event record, nErrors 2 
 HRR is STRONGLY RECOMMENDED. 
 
HRR cleared it, but we paged expert.</pre> 
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469' target=_top>193469</a>) -->
<!-- Comment: Tue Feb  8 20;35;06 comment by...Geumbong -->
Missed storable banks were TC2D, FDAD, CLAD, CLTD.


<!-- Comment: Tue Feb  8 20;40;25 comment by...Bellinger -->
Ray said that 4 events out of 200,000 wasn't really bad.  (out of 385 events processed by DAQErrMon)  However, if doing HRR didn't stop the bursts of errors, that would be a problem.  We got another burst of errors shortly thereafter, and are waiting to see if they keep coming.
<!-- Date: Tue Feb 8 20:38:53.589 2005 -->
We also got same error pattern... SCPU_BAD_VRB_BYTE_COUNT error with SCPU b0eb15 starts and daqerrmon error follows. We will count how often we have those errors.  
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469' target=_top>193469</a>) -->


<!-- Date: Tue Feb 8 20:59:07.248 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469">193469</a>
 Terminated at 2005.02.08 20:58:46
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 20:59:08.583 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469">193469</a>
TERMINATE: RC frozen.
<!-- Author: Geumbong (2080) -->



<!-- Date: Tue Feb 8 21:06:20.457 2005 -->
Got two L2 TO at 20:54 and 20:55. After that we got busy TO : VRB_ISL_06. During HRR, RC got frozen. We aborted run. 
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469' target=_top>193469</a>) -->
<!-- Comment: Tue Feb  8 21;11;56 comment by...Geumbong -->
RC can not go through idle from start. We see same symptom with previous case. We tried STOP and START L3 Proxy, but it doesn't help. 
<!-- Comment: Tue Feb  8 21;37;53 comment by...Geumbong -->
It was not the same problem as before. We checked SCPU and reset b0eb10. I'll look through ace training again. 
<!-- Date: Tue Feb 8 21:06:31.681 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2235&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2235  width='300' align='top'></a></td><td>Another "funny" cell: <b><font color=pink>B2W4L4</b></font></td></tr></table>
<!-- Author: Cristobal -->



<!-- Date: Tue Feb 8 21:14:50.862 2005 -->
<font color=green><b>CPR alarm</b> went off again.</font><br> Self-fixed when opened window in iFix.
<!-- Author: Cristobal -->


<!-- Date: Tue Feb 8 21:17:19.344 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2236&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2236' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2237&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2237' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2238&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2238' width='300'></a></center>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 21;20;36 comment by...Cristobal -->
<h3>Hourly plots</h3><pre><font color=brown><b>      20:00</b> to <b>21:00</b></font></pre>

<!-- Date: Tue Feb 8 21:25:27.014 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470">193470</a>
 Activated at 2005.02.08 21:25:21
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 21:26:21.673 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470">193470</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,535] 
<!-- Author: Geumbong (2080) -->



<!-- Date: Tue Feb 8 21:28:11.739 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2239&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2239  width='300' align='top'></a></td><td>This is the same cell as before. I unmarked, but it went pink again. Then we started taking data and HRRed, but it is still pink</td></tr></table>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 21;30;50 comment by...Cristobal -->
<br>and jet anohter time: <br><b><font color=pink>B5W6L2</font></b>

<!-- Comment: Tue Feb  8 21;32;36 comment by...Cristobal -->
after some time running, it is now going <font color=red><b>DOWN</b></font>...

<!-- Comment: Tue Feb  8 21;32;56 comment by...Cristobal -->
expert has been paged


<!-- Comment: Tue Feb  8 21;49;49 comment by...Cristobal -->
<br><i>Expert</i> had a look:<br>
-she <b><font color=yellow>unmarked</font></b> the cell but didn't change the limits<br>
-she asked for an <b><font color=red>updated</font></b> plot to be posted in two hours (or before if necessary)


<!-- Date: Tue Feb 8 21:56:49.673 2005 -->
<h3><font color=green>CPR alarm</h3></font> auto-back to normal
<!-- Author: Cristobal :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470' target=_top>193470</a>) -->


<!-- Date: Tue Feb 8 21:57:40.371 2005 -->
Busy TO for b0svx06. HRR helped it. 
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470' target=_top>193470</a>) -->


<!-- Date: Tue Feb 8 22:16:18.045 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2240&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2240' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2241&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2241' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2242&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2242' width='300'></a></center>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 22;17;29 comment by...Cristobal -->
Hourly plots: <b><font color=orange>21:00 to 22:00</b></font>


<!-- Date: Tue Feb 8 22:29:13.350 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2243&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2243  width='300' align='top'></a></td><td>again <b><font color=magenta>B2W4L4</b></font> <br>
-Unmarked, but turned to pink again <br>
-HRRed <br>
-HRRed again <br>
-finally back within acceptable limits, unmarked again and back to <b><font color=green>green</b></font></td></tr></table>
<!-- Author: Cristobal -->


<!-- Date: Tue Feb 8 23:13:34.023 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2244&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2244' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2245&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2245' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2246&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2246' width='300'></a></center>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 23;15;33 comment by...Cristobal -->
Hourly plot <font color=magenta><b>22:00</b> to <b>23:00</b></font>


<!-- Date: Tue Feb 8 23:18:57.419 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2247&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2247  width='300' align='top'></a></td><td><b><font color=red>Silicon expert</font></b> asked to see this plot:<br> 
history of B5W6L2 after going <b><font color=magenta>pink</font></b> about 2  hours ago.</td></tr></table>
<!-- Author: Cristobal -->


<!-- Date: Tue Feb 8 23:28:24.199 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2248&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2248  width='300' align='top'></a></td><td>see extrange behaviour of B2W4L4 going to <b><font color=magenta>pink</font></b><br> 
after HRR went back to <b><font color=green>normal value</font></b>. Unmarked</td></tr></table>
<!-- Author: Cristobal -->


<!-- Date: Tue Feb 8 23:43:32.180 2005 -->
<pre>Probably it's good time to summarize SCPU b0ebXX problem.  
 
 20:41, 20:51 - b0eb11 
 21:43 - b0eb16, 21:48 - b0eb14 
 22:00 to now - b0eb15(4), b0eb16(2), b0eb11(10) 
 
We also got daqerrmon error twice. </pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=103470' target=_top>103470</a>) -->


<!-- Date: Tue Feb 8 23:45:14.574 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2249&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2249  width='300' align='top'></a></td><td><b><font color=magenta>B2W5L1 pink</font></b> for a litle time</td></tr></table>
<!-- Author: Cristobal -->
<!-- Comment: Tue Feb  8 23;45;46 comment by...Cristobal -->
no HRR needed


<!-- Date: Tue Feb 8 23:50:04.360 2005 -->
We also got the busy TO : SVX06 and two L2TOs just 10 
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470' target=_top>193470</a>) -->


<!-- Date: Tue Feb 8 23:51:19.646 2005 -->
<pre> 
Attention!!!. Event Builder SCPU_CANT_RESET_VRB Error !!! 
 THE BAD VRB CRATE is: b0eb11 
 THE BAD VRB MODULE is in SLOT: 14 
 -> Halted the run and reset b0eb11, slot 14. 
</pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470' target=_top>193470</a>) -->

<!-- Date: Tue Feb 8 23:55:33.131 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470">193470</a>
 Terminated at 2005.02.08 23:55:11
<!-- Author: RunControl -->


<!-- Date: Tue Feb 8 23:55:34.631 2005 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</th>
<TH>Data Type</th>
<TH>Physics Table</th>
<TH>Begin Time</th>
<TH>End Time</th>
<TH>Live Time</th>
<TH>L1 Accepts</th>
<TH>L2 Accepts</th>
<TH>L3 Accepts</th>
<TH>Live Lumi, nb-1</th>
<TH>GR</th>
<TH>SC</th>
<TH>RC</th></tr>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193464>
193464</a> 
x2F3B8</td>                                                             
<TD>BEAM</td>                                                           
<TD>PHYSICS_3_00 [2,566,535]</td>                                       
<TD ALIGN=CENTER>15:23:13</td>                                          
<TD ALIGN=CENTER>16:59:57</td>                                          
<TD ALIGN=CENTER>01:14:01</td>                                          
<TD ALIGN=RIGHT>     89,107,965</td>                                    
<TD ALIGN=RIGHT>     1,221,466</td>                                     
<TD ALIGN=RIGHT>     303,258</td>                                       
<TD ALIGN=RIGHT>  190.837</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>1</td>                                                 
<TD ALIGN=CENTER>1</td></tr>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193465>
193465</a> 
x2F3B9</td>                                                             
<TD>BEAM</td>                                                           
<TD>PHYSICS_3_00 [2,566,535]</td>                                       
<TD ALIGN=CENTER>17:11:23</td>                                          
<TD ALIGN=CENTER>17:14:53</td>                                          
<TD ALIGN=CENTER>00:00:00</td>                                          
<TD ALIGN=RIGHT>              0</td>                                    
<TD ALIGN=RIGHT>             0</td>                                     
<TD ALIGN=RIGHT></td>                                                   
<TD ALIGN=RIGHT>    0.000</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>1</td>                                                 
<TD ALIGN=CENTER>0</td></tr>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193466>
193466</a> 
x2F3BA</td>                                                             
<TD>BEAM</td>                                                           
<TD>PHYSICS_3_00 [2,566,535]</td>                                       
<TD ALIGN=CENTER>17:40:55</td>                                          
<TD ALIGN=CENTER>18:20:24</td>                                          
<TD ALIGN=CENTER>00:17:34</td>                                          
<TD ALIGN=RIGHT>     16,457,544</td>                                    
<TD ALIGN=RIGHT>       179,409</td>                                     
<TD ALIGN=RIGHT>      57,426</td>                                       
<TD ALIGN=RIGHT>   40.181</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>1</td>                                                 
<TD ALIGN=CENTER>1</td></tr>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193467>
193467</a> 
x2F3BB</td>                                                             
<TD>BEAM</td>                                                           
<TD>PHYSICS_3_00 [2,566,535]</td>                                       
<TD ALIGN=CENTER>18:23:55</td>                                          
<TD ALIGN=CENTER>18:41:24</td>                                          
<TD ALIGN=CENTER>00:13:24</td>                                          
<TD ALIGN=RIGHT>     11,511,232</td>                                    
<TD ALIGN=RIGHT>       130,363</td>                                     
<TD ALIGN=RIGHT>      41,409</td>                                       
<TD ALIGN=RIGHT>   29.611</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>1</td>                                                 
<TD ALIGN=CENTER>1</td></tr>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193468>
193468</a> 
x2F3BC</td>                                                             
<TD>BEAM</td>                                                           
<TD>PHYSICS_3_00 [2,566,535]</td>                                       
<TD ALIGN=CENTER>18:52:05</td>                                          
<TD ALIGN=CENTER>19:29:23</td>                                          
<TD ALIGN=CENTER>00:26:59</td>                                          
<TD ALIGN=RIGHT>     27,540,321</td>                                    
<TD ALIGN=RIGHT>       315,318</td>                                     
<TD ALIGN=RIGHT>      92,027</td>                                       
<TD ALIGN=RIGHT>   57.597</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>1</td>                                                 
<TD ALIGN=CENTER>1</td></tr>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193469>
193469</a> 
x2F3BD</td>                                                             
<TD>BEAM</td>                                                           
<TD>PHYSICS_3_00 [2,566,535]</td>                                       
<TD ALIGN=CENTER>19:33:05</td>                                          
<TD ALIGN=CENTER>20:58:46</td>                                          
<TD ALIGN=CENTER>01:15:33</td>                                          
<TD ALIGN=RIGHT>     99,125,310</td>                                    
<TD ALIGN=RIGHT>     1,123,599</td>                                     
<TD ALIGN=RIGHT>     293,065</td>                                       
<TD ALIGN=RIGHT>  152.687</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>1</td>                                                 
<TD ALIGN=CENTER>1</td></tr>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470>
193470</a> 
x2F3BE</td>                                                             
<TD>BEAM</td>                                                           
<TD>PHYSICS_3_00 [2,566,535]</td>                                       
<TD ALIGN=CENTER>21:25:21</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>02:10:16</td>                                          
<TD ALIGN=RIGHT>    160,826,443</td>                                    
<TD ALIGN=RIGHT>     1,811,008</td>                                     
<TD ALIGN=RIGHT>     473,465</td>                                       
<TD ALIGN=RIGHT>  233.809</td>                                          
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER></td>                                                  
<TD ALIGN=CENTER>1</td></tr>                                            
                                                                        
<TR><TD>Totals</td>                                                     
<TD> </td>                                                              
<TD> </td>                                                              
<TD> </td>                                                              
<TD>23:55:03</td>                                                       
<TD ALIGN=CENTER>05:37:49</td>                                          
<TD ALIGN=RIGHT>     404,568,815</td>                                   
<TD ALIGN=RIGHT>      4,781,163</td>                                    
<TD ALIGN=RIGHT>     1,260,650</td>                                     
<TD ALIGN=RIGHT>  704.721</td>                                          
<TD> </td>                                                              
<TD> </td>                                                              
<TD> </td></tr>                                                         
                                                                        
</table>
<!-- Author: End of Shift Report -->



<!-- Date: Tue Feb 8 23:55:59.928 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2250&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2250  width='300' align='top'></a></td><td>Gone <b><font color=magenta>pink</font></b> again:<br> 
<h3>B5W6L2</h3></td></tr></table>
<!-- Author: Cristobal -->

<!-- Date: Tue Feb 8 23:56:54.924 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470">193470</a>
TERMINATE: end run.
<!-- Author: Geumbong (2080) -->


<!-- Date: Tue Feb 8 23:58:09.401 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193471">193471</a>
TERMINATE: try new run
<!-- Author: Geumbong (2080) -->


<!-- Date: Tue Feb 8 23:58:10.435 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193471">193471</a>
TERMINATE: end run
<!-- Author: Geumbong (2080) -->


<!-- Date: Wed Feb 9 00:07:22.046 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193472">193472</a>
 Activated at 2005.02.09 00:05:36
<!-- Author: RunControl -->


<!-- Date: Wed Feb 9 00:07:23.628 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193472">193472</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,535] 
<!-- Author: yoshiaki (2080) -->



<!-- Date: Wed Feb 9 00:07:44.452 2005 -->
<font color=red>Shift Summary: </font><pre> 
16:00 Inherited store 3972 
 
16:24 CMX miniskirt 16-17 tripped once 
 
(17:00 Main Injecter Quad power supply Circuit Breaker blown out.) 
 
17:00 Terminated run because of shepherd b0tsi00 
      Could not start run because L3proxy lost connection with RT server. 
 
17:41 Data taking resumed after L3 expert fixed the problem 
 
17:47 Data taking suspended because of ISL cooling alarm (-2.2C) 
 
18:01 Data taking resumed  
 
21:10 Restarted run after HRR failed. Needed to reset EVB crate with help from expert. 
 
21:25 Data taking resumed 
  
Major downtime 
- L3proxy lost connection with RT server (~40 minutes) 
- ISL cooling alarm (~15 minutes) 
- EVB crate reset (>15 minutes) 
- several times b0tsi crates not responding to RC, had to restart runs.  </pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   193464-193472
Delivered Luminosity    994.5 nb-1  
Acquired Luminosity     620.7 nb-1  
Efficiency              62.4%
   
</td></tr></table></pre>
<!-- Author: Xin -->


<!-- Date: Wed Feb 9 00:07:54.135 2005 -->
<pre>This time, we got  
 
 Attention!!!. SCPU_BAD_VRB_BYTE_COUNT Error !!! 
 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb24. 
 
it followed by SVX06 busy TO. At same time, we got hundreds of VRB errors. HRR failed. As recommended by error message, we end run and tried cleanup evb. 
</pre>
<!-- Author: Geumbong :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193470' target=_top>193470</a>) -->


<!-- Date: Wed Feb 9 00:09:29.215 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2251&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2251  width='300' align='top'></a></td><td>begining of run pinky<br> <b><font color=magenta>B2W4L4</font></b></td></tr></table>
<!-- Author: Cristobal -->


<!-- Date: Wed Feb 9 00:12:38.559 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2252&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2252  width='300' align='top'></a></td><td>B5W6L2 again, when not taking data it go high</td></tr></table>
<!-- Author: Cristobal -->