<!-- Title: Owl shift. Fri Aug  1, 2008 -->
<!-- SciCo: Matthew Jones -->
<!-- DAQAce: Muhammed Hussein -->
<!-- MonAce: None -->
<!-- CO: Kyung Kwang Joo -->
<!-- OpManager: Peter Dong -->
<!-- Notes: -->
<pre>Store 6326 just went in...</pre>


<!-- Date: Fri Aug 1 00:13:34.177 2008 -->
<center><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3903&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3903' width='300'></a><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3904&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3904' width='300'></a></center>
<!-- Author: mohammaf -->


<!-- Date: Fri Aug 1 00:13:40.363 2008 -->
<center><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3903&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3903' width='300'></a><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3904&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3904' width='300'></a></center>
<!-- Author: mohammad -->


<!-- Date: Fri Aug 1 00:21:02.495 2008 -->
The e-log is back after an advanced editing mishap...
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 00:21:54.328 2008 -->
Name of remote CO : Kyung Kwang Joo
<!-- Author: K.K. Joo :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 00:27:42.689 2008 -->
L2 Timeout twice. auto HRR fixed it. 
Done Timeout -simultaneously- for crates b0imu00,01,b0cmu00,01,b0cmp00,b0musc00,b0cmx00,01,b0htdc00,etcd00,clc02 and b0svt00. Auto HRR fixed this.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->
<!-- Comment: Fri Aug  1 00;38;06 comment by...mohammad -->
In the meantime we had Done Timeout in the following crates: IMU_00,IMU_01,CMU_01,CMU_00,CMP_00,MUON_SCINT_00,CMX_00,CMX_01,HADRON_TIMING_00,EM_TIMING_00,CLC_02

<!-- Date: Fri Aug 1 00:37:50.852 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265565>265565</A>
general: <tt>OK</tt>
<!-- Author: Consumer Operator( Kyung Kwang Joo) -->




<!-- Date: Fri Aug 1 00:41:52.966 2008 -->
pcal11 failed. shepherded it.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->
<!-- Comment: Fri Aug  1 01;18;49 comment by...mohammad -->
its pcal02 this time.
<!-- Date: Fri Aug 1 00:46:45.742 2008 -->
<table border=0><td><tr><td bgcolor=#e0e030>Request to the ACES:  please keep an eye on the L2 logfile.  If you see a lot of messages about XFT bunch counter/buffer number mismatches scrolling up the screan very very rapidly, please issue an HRR.  This problem likely only occurs at higher luminosity (> ~200E30).  Thanks! 
</td></tr></td></table>
<!-- Author: Heather -->
<!-- Comment: Fri Aug  1 00;56;11 comment by...mohammad -->
sure, however its scrolling slowly at the moment. 

<!-- Comment: Fri Aug  1 01;03;05 comment by...Heather -->
That's good.  This is a request to check the L2 logfile screen periodically, and if you see it scrolling very quickly to issue the HRR.  This does not happen every run.  
<br><br>
Soon we will implement the autoHRR, and you won't have to worry about it.


<!-- Comment: Fri Aug  1 01;59;54 comment by...mohammad -->
It is just fine.
<!-- Comment: Fri Aug  1 02;59;35 comment by...mohammad -->
its fine
<!-- Comment: Fri Aug  1 03;58;41 comment by...mohammad -->
ok
<!-- Comment: Fri Aug  1 06;26;37 comment by...mohammad -->
ok
<!-- Date: Fri Aug 1 01:08:44.068 2008 -->
RXPT error for svt00, auto HRR didnt help. HRR manually. In the meantime we had a L2 timeout.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->
<!-- Comment: Fri Aug  1 07;42;43 comment by...mohammad -->
RXPT for svt00. auto HRR helped this time.
<!-- Date: Fri Aug 1 01:39:01.100 2008 -->
Alarm from TOF. Every thing was ok. It went away right after the alarm! It seems a fake alarm.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 01:44:34.640 2008 -->
ICICLE icon turns red in the daqmon screen. I checked the PC and there were two weird messages. restarting it. the icon is green now. In the meantime Cellid L/B0/W0/L0/C0-1 was stuck. auto HRR fixed this.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->
<!-- Comment: Fri Aug  1 06;59;06 comment by...mohammad -->
restarting icicle again.
<!-- Date: Fri Aug 1 01:48:35.707 2008 -->
CRYO alarm. Tank 18 Ln2 Pressure is almost matching the LO Limit. 
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->



<!-- Date: Fri Aug 1 02:11:39.797 2008 -->
CER_SVXMON_HALT_RECOVER_RUN_ERROR: Pipeline out of synch in 502 silicon readout chips. Auto HRR fixed this.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->

<!-- Date: Fri Aug 1 02:19:40.183 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265565>265565</A>
general: <tt>OK</tt>
<!-- Author: Consumer Operator( Kyung Kwang Joo) -->




<!-- Date: Fri Aug 1 03:41:08.960 2008 -->
encp is stuck in ProcMon.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->
<!-- Comment: Fri Aug  1 03;51;25 comment by...mohammad -->
It is green now.


<!-- Date: Fri Aug 1 04:13:07.917 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265565>265565</A>
general: <tt>OK</tt>
<!-- Author: Consumer Operator( Kyung Kwang Joo) -->



<!-- Date: Fri Aug 1 04:31:18.088 2008 -->
All the wedges in CES,CMU,CMP,CMX and BSU were tripped and the Heartbeat boxes turn purple while both RC and L3 display were ok. We got a bunch of pup up messages in the iFIX window and in port7 PC. Few seconds later, every thing came back to normal.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 04:44:17.337 2008 -->
Heart-beat alarms on CLC and CPR iFiX nodes.  The CLC screen had a message saying that hv_to_mcr.exe had crashed.  Clicked OK... now trying to restart the monitoring programs.
<!-- Author: M Jone :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:00:31.233 2008 -->
Repeated attempts to restart the CLC monitoring keeps ending 
up with it back in the 'monitoring is STOPPED' state.
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:14:06.759 2008 -->
Called Mary - She thinks the CAEN processor might be hung.  Called MCR to let them know that we may have to cycle the CLC HV.  Mohammad has gone downstairs to inspect the CLC CAEN and possibly 
push the reset button.
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:21:14.936 2008 -->
The CAEN crate was certainly NOT hung... it responded to front-panel buttons.  Re-starting the CLC monitoring PC (did not power cycle it).
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:23:18.742 2008 -->
The PC did not restart...  Mohammad has gone downstairs to push the reset button.
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:25:49.583 2008 -->
The reset button didn't work... now power cycling it. 
Furthermore, CES,CMU,CMX,BMU,CMP just went unreachable (again).
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:31:05.860 2008 -->
CMU,CMX,etc... had messages on the screen saying something about an internal iFix error and not being able to allocate sharable memory.  But on the bright side, power cycling the CLC PC got it restarted.  Mohammad is trying to restart the ifix programs now.
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:37:59.487 2008 -->
The CLC and CPR monitoring is running now.  But the CMU,CMP,etc...  
monitoring is still not running.  But the trigger inhibit is cleared so we are taking data again.  Next, bring back the muon HV monitoring.
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->
<!-- Comment: Fri Aug  1 06;09;44 comment by...mohammad -->
Starting HVMON was Ok, the CES,CMU... boxes turned green but the H boxes kept pinky. Following the instruction for the case ¨Restarting HVMon++" and we are waiting the aftermath....
<!-- Comment: Fri Aug  1 06;24;14 comment by...mohammad -->
Restarting didnt help so I killed iFIX and restarted it. This ¨cheerfully¨ solved the problem. 
<!-- Date: Fri Aug 1 05:43:42.573 2008 -->
To keep us awake, we just got a transient solenoid alarm.  Apparently the NMR probe reading is slightly out of whack...
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 05:59:23.920 2008 -->
Muon HV monitoring eventually started and we got the heart beat back...  but wait!  now they're red... no... now they're green again except for BMU, which is yellow.  No... now they're all red again.  I'm not convinced this is normal or healthy.  There are repeated "EdaSuccess - Can't deal with tag" messages on the log 
window.  Is any of this expected?  Meanwhile, we continue to take data.
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->




<!-- Date: Fri Aug 1 06:29:39.726 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265565>265565</A>
general: <tt>OK</tt>
<!-- Author: Consumer Operator( Kyung Kwang Joo) -->



<!-- Date: Fri Aug 1 06:40:16.360 2008 -->
Someone needs to call the cryo guys between stores to reboot "that SVX computer"...  they seem to know what this is about.
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->



<!-- Date: Fri Aug 1 07:13:16.953 2008 -->
Two Gas Alarm, the Alcohol icon is red. 
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 07:17:17.405 2008 -->
Done timeout for cot16. auto HRR fixed it.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 07:33:11.869 2008 -->
Plug HV 03 tripped. Kill and reload the monitoring. 
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 07:33:23.285 2008 -->
<pre>Two ISL pinkies to note: 
  VBIAS ISL B2W1L4 just drifted slightly high. 
  DVDD ISL B0W1L4 jumped from 340 --&gt; 514 but came back down again. 
We can't save the plots because Java has run out of memory...</pre>
<!-- Author: M Jones :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 07:55:12.280 2008 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>RunNumber</TH>
<TH>DataType</TH>
<TH>PhysicsTable</TH>
<TH>Begin</TH>
<TH>End</TH>
<TH>LiveTime</TH>
<TH>L3Accepts</TH>
<TH>DelivLumi nb<SUP>-1</SUP></TH>
<TH>LiveLumi nb<SUP>-1</SUP></TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=265582 target=_runSummary>265582</A> x40D6E</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [9,1115,725]</FONT></TD>
<TD ALIGN=CENTER>00:03:12</TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>06:34:50</TD>
<TD ALIGN=RIGHT>   3,426,379</TD>   <TD ALIGN=RIGHT>  4,622.81</TD>
<TD ALIGN=RIGHT>  3,906.66</TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD>Totals</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD>
<TD>07:55:01</TD>
<TD ALIGN=CENTER>06:34:50</TD>
<TD ALIGN=RIGHT>     3,426,379</TD> <TD ALIGN=RIGHT>  4,622.81</TD>
<TD ALIGN=RIGHT>  3,906.66</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD></TR>

</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Fri Aug 1 08:06:16.660 2008 -->
<pre>b0cot12 in Error. Fails transition to halt state. 
Shepherd b0cot12 - failed. 
Shepherd b0cot12 - success.</pre>
<!-- Author: David :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 08:07:03.229 2008 -->
<pre>L2 Decision Timeout. Auto HRR. 
L2TO. Manual HRR. 
</pre>
<!-- Author: David :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 08:07:43.709 2008 -->
Error Handler is not displaying text properly.
<!-- Author: David :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=265582' target=_top>265582</a>) -->


<!-- Date: Fri Aug 1 08:11:16.882 2008 -->
<font color=red>Shift Summary: </font><pre>smooth running except for CLC iFix PC problem.</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   265582
Delivered Luminosity   4659.2  
Acquired Luminosity    3934.6  
Efficiency             84.4
   
</td></tr></table></pre>
<!-- Author: M Jones -->