<!-- Title: Eve shift. Wed Feb 16, 2005 -->
<!-- SciCo: Soo-Bong Kim -->
<!-- DAQAce: Xavier Portell Bueso -->
<!-- MonAce: Elliot Lipeles -->
<!-- CO: Carlos Sanchez -->
<!-- OpManager: Sungwon Lee -->
<!-- Notes: -->
<pre>Inherited store. Luminosity: 27E30</pre>


<!-- Date: Wed Feb 16 16:13:27.601 2005 -->
<pre>SCPU_TRACER error in b0eb11</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193991' target=_top>193991</a>) -->
<!-- Comment: Wed Feb 16 16;28;49 comment by...Xavier  -->
<pre>Same error, same b0eb11.</pre>

<!-- Comment: Wed Feb 16 16;30;42 comment by...Xavier -->
<pre>Same error, same b0eb11. Note the short difference in time.</pre>

<!-- Comment: Wed Feb 16 16;39;28 comment by...Xavier -->
<pre>Same error, now b0eb15</pre>

<!-- Comment: Wed Feb 16 16;43;43 comment by...Xavier -->
Same error, now b0eb11

<!-- Comment: Wed Feb 16 16;45;33 comment by...Xavier -->
Same error, now b0eb16. Note that now is very often that we obtain these errors (though in different modules...)


<!-- Comment: Wed Feb 16 16;58;35 comment by...Xavier -->
Same error, now b0eb14 followed by a DAQERRMON data missing error (I guess it is related)
<!-- Comment: Wed Feb 16 17;18;49 comment by...Xavier -->
Same SCPU_TRACER error in b0eb11
<!-- Comment: Wed Feb 16 17;49;37 comment by...Xavier -->
Same error, same b0eb11.
<!-- Comment: Wed Feb 16 17;53;17 comment by...Xavier -->
Same error, same crate b0eb11. Together with a busy time out.
<!-- Comment: Wed Feb 16 18;22;55 comment by...Xavier -->
Same error, same b0eb11
<!-- Date: Wed Feb 16 16:46:48.905 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2564&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2564' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2565&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2565' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2566&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2566' width='300'></a></center>
<!-- Author: lipeles -->
<!-- Comment: Wed Feb 16 16;47;21 comment by...lipeles -->
Generally cleaner, but we are still seeing small spikes.




<!-- Date: Wed Feb 16 17:33:37.377 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2569&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2569' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2567&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2567' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2568&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2568' width='300'></a></center>
<!-- Author: lipeles -->




<!-- Date: Wed Feb 16 17:57:18.797 2005 -->
SCPU_BAD_VRB_BYTE_COUNT error in b0eb14
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193991' target=_top>193991</a>) -->
<!-- Comment: Wed Feb 16 18;10;10 comment by...Xavier -->
Same error, now in b0eb11

<!-- Date: Wed Feb 16 18:21:29.275 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193991">193991</A>
 Terminated at 2005.02.16 18:21:06
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 18:21:56.518 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193991">193991</A>
TERMINATE: Need to cleanup EVB (b0tsi03 problem)
<!-- Author: Xavier Portell(x2080) -->



<!-- Date: Wed Feb 16 18:28:41.309 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993">193993</A>
 Activated at 2005.02.16 18:28:17
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 18:29:03.885 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993">193993</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] 
<!-- Author: Xavier Portell(x2080) -->



<!-- Date: Wed Feb 16 18:31:12.570 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2570&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2570' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2571&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2571' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2572&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2572' width='300'></a></center>
<!-- Author: lipeles -->
<!-- Comment: Wed Feb 16 18;31;39 comment by...lipeles -->
small jump in L1COLI
<!-- Date: Wed Feb 16 18:31:34.925 2005 -->
<pre>After having an SCPU_TRACER error on b0eb11, I got a  
crate/s: b0tsi03(138),  busy 
followed by the message: 
Host b0eb10.fnal.gov, task tsmDisp 
SM-P0-E-ScpuTimeout: SCPU ack timeout for Recover operation. --&gt; 
 Additional Information: 
 
 Attention!!!. Scanner Manager SM_SCPU_TIMEOUT Error !!!. 
 We STRONGLY RECOMMEND the following steps: 
 - Try Halt-Recover-Run. 
 - If it fails, do End (Abort) -&gt; Reset -&gt; Cleanup EVB. 
 - Start run again. 
 
As HRR didn't help (two attempts instead of one and still the same error), we end the run and clean up the EVB (no errors observed). Now it's running again.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193991' target=_top>193991</a>) -->



<!-- Date: Wed Feb 16 18:48:34.906 2005 -->
<pre>SCPU_TRACER error in b0eb15 (forwarded by FER crate COT_15). HRR solved.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993' target=_top>193993</a>) -->
<!-- Comment: Wed Feb 16 19;31;29 comment by...Xavier -->
Same error. Now b0eb11 (as in most of times of previous run)
<!-- Comment: Wed Feb 16 19;35;07 comment by...Xavier -->
Again same error and again in same place (b0eb11) (FER crate: NOT_AVAILABLE)
<!-- Comment: Wed Feb 16 19;39;39 comment by...Xavier -->
Same error but in b0ev14 (FER not available also)
<!-- Date: Wed Feb 16 18:56:53.397 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-2573&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-2573  width='300' align='top'></a></td><td>Found new problem in IMU HAT plot. East IMA/IMB wedge 15. </td></tr></table>
<!-- Author: Carlos Sanchez -->
<!-- Comment: Wed Feb 16 19;18;46 comment by...Mitch Soderberg -->
<pre>After talking to Eric about this, we're pretty sure this is a hot tower and not a TDC problem.  We'll send an e-mail to the Hadron TDC folks to let them know about this new wedge and see if anything can be done about it on an access.  In the meantime, this run should NOT be marked bad because of this.  </pre>
<!-- Date: Wed Feb 16 19:13:24.970 2005 -->
Done TO in b0imu00. HRR didn't fix it. I shepherd it twice (during the first one, b0mutr00 has become red but recovered). 
b0imu00 still not fixed so we rebooted using VISIONdemo before calling the expert. It worked!
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993' target=_top>193993</a>) -->




<!-- Date: Wed Feb 16 19:33:02.028 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2574&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2574' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2575&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2575' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2576&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2576' width='300'></a></center>
<!-- Author: lipeles -->



<!-- Date: Wed Feb 16 19:41:48.319 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993">193993</A>
 Terminated at 2005.02.16 19:41:04
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 19:41:49.837 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993">193993</A>
TERMINATE: tsi03 busy TO. Need to cleanup EVB
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 19:49:44.146 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996">193996</A>
 Activated at 2005.02.16 19:48:20
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 19:49:45.685 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996">193996</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] 
<!-- Author: Xavier Portell(x2080) -->



<!-- Date: Wed Feb 16 19:49:45.602 2005 -->
<pre>After the SCPU_TRACER problem in b0eb14, it has been a TO in b0tsi03. HRR didn't fix. We followed the instructions and ended run and cleanup EVB.  
I paged expert because it is second time that it happens during this shift...</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993' target=_top>193993</a>) -->
<!-- Comment: Wed Feb 16 19;53;55 comment by...Alberto -->
<pre>L3 Pager
Nothing peculiar to report. When I came in, EVB was ready to start a new run, after the clean-up.</pre>
<!-- Date: Wed Feb 16 19:51:36.705 2005 -->
Just started new run there has been a CER_SVXMON that HRR fixed.
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996' target=_top>193996</a>) -->



<!-- Date: Wed Feb 16 20:22:16.593 2005 -->
<pre>SCPU_BAD_VRB_BYTE_COUNT error in b0eb14. It was caused by EVB VrbBadByteCount error in SCPU=b0eb14 slot=12.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996' target=_top>193996</a>) -->


<!-- Date: Wed Feb 16 20:42:49.907 2005 -->
<pre>SCPU_TRACER error in b0eb16 (CCAL_03)</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996' target=_top>193996</a>) -->
<!-- Comment: Wed Feb 16 20;46;07 comment by...Xavier -->
Same error but in b0eb11 (crate NOT_AVAILABLE)

<!-- Comment: Wed Feb 16 20;50;43 comment by...Xavier -->
same error in same module


<!-- Comment: Wed Feb 16 21;29;19 comment by...Xavier -->
Same error, same module.
<!-- Comment: Wed Feb 16 21;59;32 comment by...Xavier -->
Same error in b0eb11 (crate CCAL_15)
<!-- Date: Wed Feb 16 21:00:45.234 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2577&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2577' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2578&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2578' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2581&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2581' width='300'></a></center>
<!-- Author: lipeles -->



<!-- Date: Wed Feb 16 22:05:05.384 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996">193996</A>
 Terminated at 2005.02.16 22:04:47
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 22:06:31.391 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996">193996</A>
TERMINATE: End the run to allow Pulsar run parasitically (with newer fer [v3_58_1]) in order to
do timing studies
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 22:07:24.801 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996">193996</A>
TERMINATE: end of run
<!-- Author: Xavier Portell(x2080) -->
<!-- Comment: Wed Feb 16 22;09;36 comment by...Xavier -->
This second entry is because it has been forgotten to click the boxes when giving the reason for the end-of-run.
<!-- Date: Wed Feb 16 22:08:03.016 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2585&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2585' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2586&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2586' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2587&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2587' width='300'></a></center>
<!-- Author: lipeles -->


<!-- Date: Wed Feb 16 22:08:45.993 2005 -->
SVT is still running (happily) with the LC cut on. I will plan on being in B0 between 8:30 and 9 am tomorrow to restore normal operations for the next store (takes about 30 seconds or less). If for some reason the Run Coordinator changes plans and a new store will start before then, please beep me (SVT pager).
<!-- Author: Jahred -->


<!-- Date: Wed Feb 16 22:11:45.054 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193997">193997</A>
 Activated at 2005.02.16 22:11:20
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 22:13:31.484 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193997">193997</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] 
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 22:28:09.431 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193997">193997</A>
 Terminated at 2005.02.16 22:27:54
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 22:28:43.223 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193997">193997</A>
TERMINATE: end of run
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 22:35:00.359 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193998">193998</A>
 Activated at 2005.02.16 22:33:35
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 22:35:01.105 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193998">193998</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] continue with Pulsar timing studies
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 22:46:46.168 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193998">193998</A>
 Terminated at 2005.02.16 22:46:22
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 22:50:23.298 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193998">193998</A>
TERMINATE: end the run to take L2TOTS board "out"
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 22:50:24.651 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193998">193998</A>
TERMINATE: end the run to take L2TOTS board "out" (fro timing measurements)
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 22:50:25.919 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193999">193999</A>
TERMINATE: taking L2TOTS out does not fly. Abort the run (at config stage).  Try to take every other (not L2TOTS) board out instead
<!-- Author: Xavier Portell(x2080) -->


<!-- Date: Wed Feb 16 22:53:14.621 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194000">194000</A>
 Activated at 2005.02.16 22:52:53
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 22:57:02.378 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194000">194000</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] start a new run, all boards except 
L2TOTS (slot 17) are disabled.
<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:06:32.387 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194000">194000</A>
 Terminated at 2005.02.16 23:06:05
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 23:07:28.402 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194000">194000</A>
TERMINATE: end the run to try yet another configuration in b0l2pu00 crate
<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:10:03.491 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194001">194001</A>
 Activated at 2005.02.16 23:09:48
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 23:11:41.172 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194001">194001</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] 
<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:13:51.451 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193991">193991</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
SVT 
</PRE>Long cluster cut included
<!-- Author: cdfscico -->



<!-- Date: Wed Feb 16 23:15:04.305 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2588&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2588' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2589&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2589' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-2590&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-2590' width='300'></a></center>
<!-- Author: lipeles -->

<!-- Date: Wed Feb 16 23:17:14.002 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993">193993</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
SVT 
</PRE>Long cluster cut included
<!-- Author: cdfscico -->


<!-- Date: Wed Feb 16 23:19:22.922 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996">193996</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
SVT 
</PRE>Long cluster cut included
<!-- Author: cdfscico -->


<!-- Date: Wed Feb 16 23:21:28.171 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194001">194001</A>
 Terminated at 2005.02.16 23:20:54
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 23:22:01.247 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194001">194001</A>
TERMINATE: end of run with L2TOTS and the two mergers read out (others disabled)
<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:27:54.516 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194002">194002</A>
 Activated at 2005.02.16 23:24:51
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 23:27:55.555 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194002">194002</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] start a run with yet another configuration of boards in b0l2pu00 crate
<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:41:19.782 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194002">194002</A>
 Terminated at 2005.02.16 23:40:36
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 23:42:21.424 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194002">194002</A>
TERMINATE: finish another test ...
<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:48:43.398 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194003">194003</A>
TERMINATE: aborted before activate

<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:50:52.196 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193997">193997</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L2T 
SVT 
</PRE>Pulsar test, Long cluster cut included
<!-- Author: cdfscico -->


<!-- Date: Wed Feb 16 23:53:49.407 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194004">194004</A>
 Activated at 2005.02.16 23:51:58
<!-- Author: RunControl -->


<!-- Date: Wed Feb 16 23:53:50.572 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193998">193998</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L2T 
SVT 
</PRE>Pulsar test, Long cluster cut included
<!-- Author: cdfscico -->


<!-- Date: Wed Feb 16 23:53:51.413 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194000">194000</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L2T 
SVT 
</PRE>Pulsar test, Long cluster cut included
<!-- Author: cdfscico -->


<!-- Date: Wed Feb 16 23:53:53.530 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194004">194004</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [2,566,536] Last 10min we will test the Pulsar implementaion of ReadOutList.  Pulsar is driving (all silicon VRB and FIBS are OUT)
<!-- Author: cjl / vrusu / anikeev x2080 -->


<!-- Date: Wed Feb 16 23:56:00.314 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194001">194001</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L2T 
SVT 
</PRE>Pulsar test, Long cluster cut included
<!-- Author: cdfscico -->


<!-- Date: Wed Feb 16 23:56:01.645 2005 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193991>
193991</A> 
x2F5C7</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>14:53:24</TD>                                          
<TD ALIGN=CENTER>18:21:06</TD>                                          
<TD ALIGN=CENTER>02:57:48</TD>                                          
<TD ALIGN=RIGHT>    192,298,446</TD>                                    
<TD ALIGN=RIGHT>     2,064,351</TD>                                     
<TD ALIGN=RIGHT>     576,875</TD>                                       
<TD ALIGN=RIGHT>  275.144</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193993>
193993</A> 
x2F5C9</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>18:28:17</TD>                                          
<TD ALIGN=CENTER>19:41:04</TD>                                          
<TD ALIGN=CENTER>00:58:53</TD>                                          
<TD ALIGN=RIGHT>     53,620,076</TD>                                    
<TD ALIGN=RIGHT>       582,870</TD>                                     
<TD ALIGN=RIGHT>     168,620</TD>                                       
<TD ALIGN=RIGHT>   80.630</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193996>
193996</A> 
x2F5CC</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>19:48:20</TD>                                          
<TD ALIGN=CENTER>22:04:47</TD>                                          
<TD ALIGN=CENTER>02:11:57</TD>                                          
<TD ALIGN=RIGHT>    119,082,019</TD>                                    
<TD ALIGN=RIGHT>     1,285,900</TD>                                     
<TD ALIGN=RIGHT>     370,096</TD>                                       
<TD ALIGN=RIGHT>  165.562</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193997>
193997</A> 
x2F5CD</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>22:11:20</TD>                                          
<TD ALIGN=CENTER>22:27:54</TD>                                          
<TD ALIGN=CENTER>00:10:54</TD>                                          
<TD ALIGN=RIGHT>      3,828,396</TD>                                    
<TD ALIGN=RIGHT>        54,270</TD>                                     
<TD ALIGN=RIGHT>      20,276</TD>                                       
<TD ALIGN=RIGHT>   12.918</TD>                                          
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=193998>
193998</A> 
x2F5CE</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>22:33:35</TD>                                          
<TD ALIGN=CENTER>22:46:22</TD>                                          
<TD ALIGN=CENTER>00:12:34</TD>                                          
<TD ALIGN=RIGHT>      4,830,677</TD>                                    
<TD ALIGN=RIGHT>        63,957</TD>                                     
<TD ALIGN=RIGHT>      23,314</TD>                                       
<TD ALIGN=RIGHT>   14.606</TD>                                          
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194000>
194000</A> 
x2F5D0</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>22:52:53</TD>                                          
<TD ALIGN=CENTER>23:06:05</TD>                                          
<TD ALIGN=CENTER>00:13:02</TD>                                          
<TD ALIGN=RIGHT>      5,155,352</TD>                                    
<TD ALIGN=RIGHT>        66,492</TD>                                     
<TD ALIGN=RIGHT>      24,055</TD>                                       
<TD ALIGN=RIGHT>   14.643</TD>                                          
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194001>
194001</A> 
x2F5D1</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>23:09:48</TD>                                          
<TD ALIGN=CENTER>23:20:54</TD>                                          
<TD ALIGN=CENTER>00:09:39</TD>                                          
<TD ALIGN=RIGHT>      3,349,052</TD>                                    
<TD ALIGN=RIGHT>        45,937</TD>                                     
<TD ALIGN=RIGHT>      16,984</TD>                                       
<TD ALIGN=RIGHT>   10.864</TD>                                          
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194002>
194002</A> 
x2F5D2</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [2,566,536]</TD>                                       
<TD ALIGN=CENTER>23:24:51</TD>                                          
<TD ALIGN=CENTER>23:40:36</TD>                                          
<TD ALIGN=CENTER>00:15:27</TD>                                          
<TD ALIGN=RIGHT>      7,099,344</TD>                                    
<TD ALIGN=RIGHT>        87,224</TD>                                     
<TD ALIGN=RIGHT>      30,221</TD>                                       
<TD ALIGN=RIGHT>   17.175</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>23:55:03</TD>                                                       
<TD ALIGN=CENTER>07:10:17</TD>                                          
<TD ALIGN=RIGHT>     389,263,362</TD>                                   
<TD ALIGN=RIGHT>      4,251,001</TD>                                    
<TD ALIGN=RIGHT>     1,230,441</TD>                                     
<TD ALIGN=RIGHT>  591.543</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Wed Feb 16 23:56:14.028 2005 -->
pink silicon cells  
DVDD B2 W2 L4 
AVDD B2 W4 L4 
AVDD B2 W2 L4 
all unmark to green
<!-- Author: lipeles -->

<!-- Date: Wed Feb 16 23:56:34.931 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194002">194002</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L2T 
SVT 
</PRE>Pulsar test, Long cluster cut included
<!-- Author: cdfscico -->



<!-- Date: Wed Feb 16 23:58:56.137 2005 -->
<font color=red>Shift Summary: </font><pre>16:00  Store 3986 colliding. Luminosity: 27E30 
18:31  SCPU_TRACER error on b0eb11. No recovery attempts worked. 
       Ended the run and started a new run. 
18:56  A problem obseved in the IMU HAT plot by CO monitoring 
       (East IMA/IMB wedge 15)  
22:00  Pulsar beam test for the next two hours</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   193991-194002
Delivered Luminosity   632.0  
Acquired Luminosity    522.0  
Efficiency             82.6
   
</td></tr></table></pre>
<!-- Author: Soo-Bong -->

<!-- Date: Thu Feb 17 00:00:56.375 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194004">194004</A>
 Terminated at 2005.02.17 00:00:25
<!-- Author: RunControl -->


<!-- Date: Thu Feb 17 00:02:05.766 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194004">194004</A>
TERMINATE: end of testing for tonight
<!-- Author: daq ace -->



<!-- Date: Thu Feb 17 07:52:47.690 2005 -->
<pre>PHA, PEM, PSH turn red in iFix.  Problems disappears on it's own while Monitoring Ace investigates.</pre>
<!-- Author: Aseet Mukherjee :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194007' target=_top>194007</a>) -->


<!-- Date: Thu Feb 17 07:57:09.410 2005 -->
<font color=red>Shift Summary: </font><pre>Started new run at start of store (ended Pulsar testing done at end of evening shift) 
 
DCAS-ADMEM comparison plot, which showed some bad regions before, seems to be worse now.  See the plot in entered by Ivan at 4:33. 
 
3:52-4:39 iFix node VNODE1 got stuck.  We had to reboot (unfortunately had to call Steve Hahn for the password).  Still wasn't comminucating with several systems.  Killed and restarted iFix, and that seemed to fix it.  No data lost due to this. 
 
4:41-5:51 Several CMX trips corrolated with spikes in lostp.  Called MCR at 5:33, they said they'd adjust the tune.  MCR claimed we could keep running while they did this.  Paged silicon to see if they were comfortable staying on; at 5:45, Sebastian said we should turn off silicon.  We  did, almost started a  NOSILICON run ... then MCR called to say they were done tuning.  We went back to normal (with silicon) running. 
 
6:18 MCR called at to say store will end at 9AM. 
 
7:47 More iFix problems ... PHA, PEM, PSH turn red, but no trigger inhibit.  We continue taking data.  Problem disappears on its own. 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   Started new run at start of store (ended Pulsar testing done at end of evening shift)

DCAS-ADMEM comparison plot, which showed some bad regions before, seems to be worse now.  See the plot in entered by Ivan at 4:33.

3:52-4:39 iFix node VNODE1 got stuck.  We had to reboot (unfortunately had to call Steve Hahn for the password).  Still wasn't comminucating with several systems.  Killed and restarted iFix, and that seemed to fix it.  No data lost due to this.

4:41-5:51 Several CMX trips corrolated with spikes in lostp.  Called MCR at 5:33, they said they'd adjust the tune.  MCR claimed we could keep running while they did this.  Paged silicon to see if they were comfortable staying on; at 5:45, Sebastian said we should turn off silicon.  We  did, almost started a  NOSILICON run ... then MCR called to say they were done tuning.  We went back to normal (with silicon) running.

6:18 MCR called at to say store will end at 9AM.

Started new run at start of store (ended Pulsar testing done at end of evening shift)

DCAS-ADMEM comparison plot, which showed some bad regions before, seems to be worse now.  See the plot in entered by Ivan at 4:33.

3:52-4:39 iFix node VNODE1 got stuck.  We had to reboot (unfortunately had to call Steve Hahn for the password).  Still wasn't comminucating with several systems.  Killed and restarted iFix, and that seemed to fix it.  No data lost due to this.

4:41-5:51 Several CMX trips corrolated with spikes in lostp.  Called MCR at 5:33, they said they'd adjust the tune.  MCR claimed we could keep running while they did this.  Paged silicon to see if they were comfortable staying on; at 5:45, Sebastian said we should turn off silicon.  We  did, almost started a  NOSILICON run ... then MCR called to say they were done tuning.  We went back to normal (with silicon) running.

6:18 MCR called at to say store will end at 9AM.

Started new run at start of store (ended Pulsar testing done at end of evening shift)

DCAS-ADMEM comparison plot, which showed some bad regions before, seems to be worse now.  See the plot in entered by Ivan at 4:33.

3:52-4:39 iFix node VNODE1 got stuck.  We had to reboot (unfortunately had to call Steve Hahn for the password).  Still wasn't comminucating with several systems.  Killed and restarted iFix, and that seemed to fix it.  No data lost due to this.

4:41-5:51 Several CMX trips corrolated with spikes in lostp.  Called MCR at 5:33, they said they'd adjust the tune.  MCR claimed we could keep running while they did this.  Paged silicon to see if they were comfortable staying on; at 5:45, Sebastian said we should turn off silicon.  We  did, almost started a  NOSILICON run ... then MCR called to say they were done tuning.  We went back to normal (with silicon) running.

6:18 MCR called at to say store will end at 9AM.

 194005 - 194007
Delivered Luminosity    445.6  
Acquired Luminosity     445.6  
Efficiency              84.9
   
</td></tr></table></pre>
<!-- Author: Aseet Mukherjee -->