<!-- Title: Day shift. Thu Oct 11, 2007 -->
<!-- SciCo: Manfred Paulini -->
<!-- DAQAce: Carolina Deluca -->
<!-- MonAce: &nbsp; -->
<!-- CO: &nbsp; -->
<!-- OpManager: Steve Hahn -->
<!-- Notes: -->
<pre>Let's start the shift and see how the day goes. Having trouble starting cosmics. Matthew Jones has wcal03 to diagnose ToF issues remotely from Purdue.
BTW, the little boss (now deputy on the 2nd floor) made appearance in the control room :-) </pre>


<!-- Date: Thu Oct 11 08:10:14.775 2007 -->
I turned the volume on the speakers all the way down.  The audible alarm was driving the shift crew crazy. 
When the problem is fixed, please put the speakers back on
<!-- Author: dee -->


<!-- Date: Thu Oct 11 08:10:38.829 2007 -->
HAPPY BIRTHDAY JJ
<!-- Author: Dee -->


<!-- Date: Thu Oct 11 08:19:33.706 2007 -->
I've been trying to take a TAC calibration and write all the raw 
data to the look area, but it appears that nothing has been 
written to the look area since October 4th.  Is the CSL working?
<!-- Author: M Jones -->
<!-- Comment: Thu Oct 11 12;05;37 comment by...Manfred -->
Made an entry further below but should have put this here:
paged CSL and Ilja told us that the Look area is being moved from one computer to another. 
<!-- Comment: Thu Oct 11 12;06;34 comment by...Manfred -->
Ilja says files are written to new area and he will contact Matthew Jones about file location.
<!-- Date: Thu Oct 11 08:20:28.857 2007 -->
<pre> RXPT Error for crate/s:  
    Busy Timeout.  
 If problem persists, please use the DiagnoseTO button  
 on the EVB Ace Control Panel. [RXPT]  
 
FrontEnd Crate Error Condition from: 
PCAL_03 
 
Done Timeout detected in crates: 
PCAL_03 
CMX_00 
 
Run is halted. Trying to shepherd the crate pcal03 which is red in partition pannel</pre>
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251383' target=_top>251383</a>) -->
<!-- Comment: Thu Oct 11 08;23;01 comment by...carolina -->
shepherding is taking too much time. It seems it cannot connect to the crate....


<!-- Date: Thu Oct 11 08:29:54.470 2007 -->
<pre>Run Control was stucked at Halt, I restarted it. 
 
I tried to connect to b0pcal03 from the terminal but it's not responding. Manfred is trying to find if someone is working on it now. 
 
Will restart cosmics run with no pcal03, no wcal crates and ignoring reces and svt.</pre>
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251383' target=_top>251383</a>) -->
<!-- Comment: Thu Oct 11 08;52;41 comment by...Manfred -->
Talked with Steve Hahn - he thinks somebody might work on the crate - we'll continue without pcal03 until Steve comes in to investigate further
<!-- Date: Thu Oct 11 08:32:33.341 2007 -->
GAS Alarm. Scico checked they are wotking on the CLC cooling.
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251383' target=_top>251383</a>) -->
<!-- Comment: Thu Oct 11 08;39;54 comment by...carolina -->
Someone silenced CLC alarms but not here in the CR
<!-- Comment: Thu Oct 11 08;40;38 comment by...Manfred -->
Bill Neu called to find out who silenced the alarm - it was not us!
<!-- Date: Thu Oct 11 08:39:21.093 2007 -->
RTServer error in Procmon. I forced another check and now it's green again.
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251385' target=_top>251385</a>) -->

<!-- Date: Thu Oct 11 08:39:54.548 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251385>251385</A>
 Activated at 2007.10.11 08:39:35
<!-- Author: RunControl -->




<!-- Date: Thu Oct 11 08:41:11.087 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251385>251385</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,954,639] no pcal03, no wcal crates. Ignoring reces and svt. As usual  Done TO. for crate/s: 
    b0svt09  As usual got  Done TO. for crate/s: svt09 but HRR recovered active state.
    b0svt09   
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 08:56:13.587 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251385>251385</A>
 Terminated at 2007.10.11 08:55:57
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 08:56:31.527 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251385>251385</A>
TERMINATE: Ended to remove ccal00 from partition as requested by Donatella
<!-- Author: Carolina Deluca -->
<!-- Comment: Thu Oct 11 09;30;45 comment by...Donatella -->
released ccal00. Tested a new Sevb with latest L2 bank changes. It seems to be working.
<!-- Date: Thu Oct 11 09:00:21.079 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251386>251386</A>
 Activated at 2007.10.11 08:59:02
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:00:22.397 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251386>251386</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,954,639] excluded from partition: ccal00, pcal03 and wcal crates. Ignoring reces and svt.
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 09:08:41.263 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251386>251386</A>
 Terminated at 2007.10.11 09:07:55
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:08:42.300 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251386>251386</A>
TERMINATE: Ended to test a new Trigger Table with L2 torture (Alison and Laura request)
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 09:19:59.135 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251392>251392</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,972,639] testing new trigger table
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 09:20:03.531 2007 -->
Matthew Jones called back and is wondering whether it is possible to write to the Look area. This question already came up a few days ago => time to investigate.
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251392' target=_top>251392</a>) -->
<!-- Comment: Thu Oct 11 09;51;38 comment by...Manfred. -->
Paged first DAQ and found out that the pager sitting on the SciCo desk for over a week is the DAQ pager. Then paged CSL and Ilja told us that the Look area is being moved from one computer to another. He also was wondering whether he can have the system for 10 min this morning.
<!-- Comment: Thu Oct 11 17;56;54 comment by...M. Datta -->
<pre>As for the DAQ pager we have thought that it is with Bill Badgett, who has been away. On Monday when I was supposed to get the pager, I told Frank Chlebana (SciCo) to contact me on my cell phone for any DAQ related problems. Frank has posted an entry on the e-log about that (Mon Oct 8 18:11:48).
Now I have picked up the DAQ pager from the control room. Sorry about the confusion.  </pre>
<!-- Date: Thu Oct 11 09:22:13.446 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251392>251392</A>
TERMINATE: It worked :-)
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 09:27:36.773 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251394>251394</A>
 Activated at 2007.10.11 09:25:49
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:27:37.296 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251394>251394</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 09:27:38.224 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251394>251394</A>
 Terminated at 2007.10.11 09:26:01
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:28:39.157 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251395>251395</A>
 Activated at 2007.10.11 09:28:31
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:28:51.168 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251395>251395</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 09:31:12.419 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251395>251395</A>
 Terminated at 2007.10.11 09:30:52
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:35:23.509 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251396>251396</A>
 Activated at 2007.10.11 09:35:08
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:41:38.456 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251396>251396</A>
 Terminated at 2007.10.11 09:39:30
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:41:39.543 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251396>251396</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 09:42:51.377 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251397>251397</A>
 Activated at 2007.10.11 09:41:41
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:42:52.334 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251397>251397</A>
 Terminated at 2007.10.11 09:42:28
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 09:48:18.392 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251398>251398</A>
 Activated at 2007.10.11 09:48:11
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 09:48:18.445 2007 -->
It looks like CMU/P/X HV made it through the night OK. That's the good news.
<!-- Author: Phil -->

<!-- Date: Thu Oct 11 09:49:08.180 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251398>251398</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 09:49:29.567 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251398>251398</A>
 Terminated at 2007.10.11 09:49:24
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 09:53:23.649 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251399>251399</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,972,639] 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 10:01:40.389 2007 -->
GAM HV turned red on iFix, but alarms are silenced. Is anyobody working on there?
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251399' target=_top>251399</a>) -->
<!-- Comment: Thu Oct 11 10;13;09 comment by...carolina -->
it went green again
<!-- Date: Thu Oct 11 10:07:58.696 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251401>251401</A>
 Activated at 2007.10.11 10:07:01
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:07:59.629 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251401>251401</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 10:09:54.829 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251401>251401</A>
 Terminated at 2007.10.11 10:08:34
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:12:00.983 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251402>251402</A>
 Activated at 2007.10.11 10:11:53
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 10:14:48.096 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251402>251402</A>
 Terminated at 2007.10.11 10:14:27
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:17:22.284 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251403>251403</A>
 Activated at 2007.10.11 10:17:15
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:18:34.668 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251403>251403</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 10:21:37.219 2007 -->
<p>Del has written me and other involved parties that both 
BMU and CLC are ready for HV.  I will now undo the last LOTO 
locks I have in place.</p>
<!-- Author: Steve Hahn :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251403' target=_top>251403</a>) -->

<!-- Date: Thu Oct 11 10:22:47.888 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251403>251403</A>
 Terminated at 2007.10.11 10:22:34
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:25:48.202 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251404>251404</A>
 Activated at 2007.10.11 10:25:34
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 10:28:27.126 2007 -->
<pre>This morning we've been testing new L2 tagsets: 972 for L2 Torture and 973 for Cosmics. 
These include the latest changes for event truncation. 
We tested a few combinations of the register settings and it did seem to truncate at the right number of words. We were able to run with every event truncated.</pre>
<!-- Author: Laura + Alison -->

<!-- Date: Thu Oct 11 10:29:12.617 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251404>251404</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 10:31:44.629 2007 -->
Jonathan fixed the audible alarm.  Thank you.  I turned up the volume.
<!-- Author: dee -->

<!-- Date: Thu Oct 11 10:32:47.603 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251404>251404</A>
 Terminated at 2007.10.11 10:32:29
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 10:32:47.169 2007 -->
Jonathan fixed the problem with the "partition" alarm. It was caused by run control using partition 6, but anyone was working on it
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251404' target=_top>251404</a>) -->

<!-- Date: Thu Oct 11 10:41:07.168 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251406>251406</A>
 Activated at 2007.10.11 10:40:36
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:42:30.636 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251406>251406</A>
 Terminated at 2007.10.11 10:41:56
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:47:38.913 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251407>251407</A>
 Activated at 2007.10.11 10:47:28
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:48:49.949 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251407>251407</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] no wcal crates. no ccal00 crate.
<!-- Author: Carolina Deluca -->
<!-- Comment: Thu Oct 11 10;50;25 comment by...Donatella T. -->
I have released the ccal00. You can include it (sorry, forgot to call back)
<!-- Date: Thu Oct 11 10:48:51.182 2007 -->
We have permission for HV on BMU and CLC.
<!-- Author: Phil Schlabach -->

<!-- Date: Thu Oct 11 10:49:33.226 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251407>251407</A>
 Terminated at 2007.10.11 10:49:18
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 10:49:33.598 2007 -->
I will be stopping the muon HV monitoring to include BMU CAENS. Then I will bring the HV on.
<!-- Author: Phil -->
<!-- Comment: Thu Oct 11 11;06;41 comment by...Phil -->
West anode 24 consistently trips a little above 1000V. West anode 156 tripped the first time, but is now at full HV. Anode 168 is drawing 16microA. That is way too much. Everything else is at full HV with normal current draw.
<!-- Date: Thu Oct 11 10:50:20.914 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251407>251407</A>
TERMINATE: need to ignore reces and svt
<!-- Author: Carolina Deluca -->




<!-- Date: Thu Oct 11 10:51:18.472 2007 -->
Start a cosmics run for Aseet, Bob and Morris to hunt down the noise problem in the COT
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408' target=_top>251408</a>) -->

<!-- Date: Thu Oct 11 10:52:32.705 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408>251408</A>
 Activated at 2007.10.11 10:51:43
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 10:52:33.423 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408>251408</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 10:52:33.230 2007 -->
the Software event builder has been updated to a new exe to test some L2 bank changes. *IF* you have any problems using it, please call me (630-482-2642) and I'll switch back to the standard one.
<!-- Author: Donatella T. :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408' target=_top>251408</a>) -->


<!-- Date: Thu Oct 11 10:54:52.200 2007 -->
Morris Binkley, Bob Wagner and Aseet Mukherjee need a cosmics run to work on COT crates. Laura also asked us to run cosmics with trigger table [15,973,639]. 
 
Donatella released ccal00 crate. We'll include in partition for next run. 
 
For this run, we don't have wcal crates and ccal00 but we recovered     
pcal03. We are also ignoring reces and svt. 
 
CLC, MNP, BCS and CPR turned red on iFix. They are turning on the HV.
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408' target=_top>251408</a>) -->



<!-- Date: Thu Oct 11 11:09:46.029 2007 -->
TWO evb2b TO: 
 
If problem persists, please use the DiagnoseTO button  
 on the EVB Ace Control Panel.  
 
2nd HRR
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408' target=_top>251408</a>) -->

<!-- Date: Thu Oct 11 11:10:10.011 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408>251408</A>
 Terminated at 2007.10.11 11:09:49
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 11:10:35.313 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251408>251408</A>
TERMINATE: COT guys want to start a new run.
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 11:13:49.270 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251409>251409</A>
 Activated at 2007.10.11 11:13:34
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 11:15:21.021 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251409>251409</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] ccal00 included again. no wcal crates in partition.
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 11:19:32.027 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251409>251409</A>
 Terminated at 2007.10.11 11:19:23
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 11:19:50.752 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251409>251409</A>
TERMINATE: forget to ignore reces and svt
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 11:22:43.939 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251410>251410</A>
 Activated at 2007.10.11 11:22:05
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 11:22:44.535 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251410>251410</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] no wcal crates, ignoring reces and svt
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 11:23:22.494 2007 -->
<pre>Done TO. for crate/s:  
    b0svt09    
 
HRR recovered active state</pre>
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251410' target=_top>251410</a>) -->


<!-- Date: Thu Oct 11 11:32:48.168 2007 -->
cdfdaq is over quota. Page JDL who comes running and takes care of the problem
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251410' target=_top>251410</a>) -->


<!-- Date: Thu Oct 11 11:33:11.074 2007 -->
<pre> 
Error: cdfdaq is over 95% of its quota: 
Disk quotas for user cdfdaq (uid 5430):  
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace 
      /dev/sda9       4       0 20000000               1       0  100000         
b0home:/vol/home 
                7655188  12582912 15728640           41536  4294967295 4294967295         
b0spool:/vol/data 
                121772348  125829120 138412032          447286  4294967295 4294967295         
b0spool:/vol/spool 
                10453376  26214400 26214400             970  4294967295 4294967295         
Percent of home quota: 60 
Percent of data quota: 96</pre>
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251410' target=_top>251410</a>) -->


<!-- Date: Thu Oct 11 11:40:20.390 2007 -->
I turned BMU HV off as I expect experts will be working this afternoon. East anode 144 tripped after 10 min. The rest held (except for the one noted above).
<!-- Author: Phil -->

<!-- Date: Thu Oct 11 11:49:43.915 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251410>251410</A>
 Terminated at 2007.10.11 11:49:35
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 11:50:41.673 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251410>251410</A>
TERMINATE: COT guys asked for end the run
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 11:51:38.431 2007 -->
Muon detector status: 
<pre> 
CMU/P/X held HV overnight except Swall a4 where there is a known problem with a stack which has been identified earlier in the shutdown. BMU has been on for 20 minutes except for West anode 24 which trips at ~1500V. West anode 168 is drawing way too much current.   East anode 144 tripped after 10 min or so. I've turned BMU off for work this PM.</pre> 
<p>
I observe the following problems from the monitoring. 
<ul> 
<li>CMP</li> 
<ul> 
<li>Top stack 1 appears to be oscillating.</li> 
<li>Bottom stack 41-43 and 48-53 are mostly dead. It looks to me like cables 7B and 8B are at issue. We can't seem to get through a shutdown without screwing up the bottom CMP crate...</li> 
</ul> 
<li>CMU</li> 
<ul> 
<li>West 5, ch. 0-31 seem to be dead.</li> 
</ul> 
<li>CMX</li> 
<ul> 
<li>west 0,1 have a bunch of noisy channels</li> 
<li>SE miniskirt appears dead.</li> 
</ul> 
</ul>
<!-- Author: Phil -->
<!-- Comment: Thu Oct 11 12;16;46 comment by...Phil, Dee -->
at least one ASD input cable is pulled out of the connector, others I had a hard time telling. NW CMX will have to be pulled out to inspect and fix it.
<!-- Comment: Thu Oct 11 12;17;06 comment by...Phil -->
That refers to the problem in bottom CMP.
<!-- Comment: Thu Oct 11 13;54;07 comment by...Phil -->
After puzzling about what I saw in the monitoring and what I saw at the ASD crate, I realized that I had the wrong mapping of channel to cable. The monitoring plots indicate 6B (stack 41-43) and 8T (stack 48-53). I observed 8T cocked out from the top. The indication is that 6B has the same orientation, but it was hidden from me as I didn't have a borescope in my back pocket.
<!-- Comment: Thu Oct 11 14;26;22 comment by...Dale Stentz and John Strologas -->
Update on muon status:<br><br>
Turning off the LV in the NW CMX (west wedges 23-1) seemd to have killed the noise.  It may return but we'll be able to debug further if it does.  The noise can be seen in YMon although looking at the event display for pink '+' in the CMX wedges around phi = 0 is actually quicker and more obvious for the curious ace.<br><br>
CMX NE (not SE) miniskirt is in a modified configuration.  We will move it back to a operational/data-taking configuration soon.
<br><Br>
With respect to CMP 6B (if we understand Phil's mapping) - Dale inspected the port card going into the ASD and can see that the top latch is broken off.  We've put in new latches before and it is possible to jury-rigg the cable to connect.  Dale left the card (6B) out as it was really out of position and straining the bottom latch (still good).  As suggested above, any real fix or improvement in the location of the cables will require the NW CMX arch out.
<!-- Comment: Thu Oct 11 22;33;12 comment by...phil -->
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2007&action=view&page=791&frame=2&anchor=22:20:44.448'>
follow up</a> from later
<!-- Date: Thu Oct 11 11:53:11.957 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251411>251411</A>
 Activated at 2007.10.11 11:53:00
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 11:54:02.353 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251411>251411</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] a new run was required by the COT guys. Ignoring reces and svt. wcal crates are out of the partition.
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 11:56:48.470 2007 -->
TrigMon filled the cdfdaq quota on the data disk (again). 
<pre> 
-rw-r--r--  1 cdfdaq cdf          0 Oct 11 11:53 TrigMon_251411_001.log 
-rw-r--r--  1 cdfdaq cdf       1165 Oct 11 11:32 TrigMon_251410_001.log 
-rw-r--r--  1 cdfdaq cdf  736364611 Oct 11 11:32 TrigMon_251408_001.log 
-rw-r--r--  1 cdfdaq cdf 1177029649 Oct 11 11:32 TrigMon_251404_001.log 
-rw-r--r--  1 cdfdaq cdf 1177031373 Oct 11 11:32 TrigMon_251402_001.log 
-rw-r--r--  1 cdfdaq cdf 1544033022 Oct 11 11:32 TrigMon_251401_001.log 
-rw-r--r--  1 cdfdaq cdf 1835536295 Oct 11 11:32 TrigMon_251396_001.log 
-rw-r--r--  1 cdfdaq cdf 2275413921 Oct 11 11:32 TrigMon_251392_001.log 
-rw-r--r--  1 cdfdaq cdf 2275866998 Oct 11 11:32 TrigMon_251386_001.log 
-rw-r--r--  1 cdfdaq cdf 2276650966 Oct 11 11:32 TrigMon_251385_001.log 
-rw-r--r--  1 cdfdaq cdf 2277495607 Oct 11 11:32 TrigMon_251383_001.log 
-rw-r--r--  1 cdfdaq cdf 2277497514 Oct 11 11:32 TrigMon_251381_001.log 
-rw-r--r--  1 cdfdaq cdf 2299830076 Oct 11 11:32 TrigMon_251380_001.log 
-rw-r--r--  1 cdfdaq cdf 2300636498 Oct 11 11:32 TrigMon_251379_001.log 
-rw-r--r--  1 cdfdaq cdf 2300637422 Oct 11 11:32 TrigMon_251372_001.log 
-rw-r--r--  1 cdfdaq cdf        409 Oct 10 23:01 TrigMon_251371_001.log 
-rw-r--r--  1 cdfdaq cdf        409 Oct 10 22:53 TrigMon_251369_002.log 
</pre> 
I'll delete a bunch of files.
<!-- Author: jdl -->


<!-- Date: Thu Oct 11 12:01:13.608 2007 -->
The following error is repeated bazillions of times: 
<pre> 
%ERLOG-e Inconsistent number of blocks and data size:  
         /cdf/onln/code/b0cdfsoft/cdfsoftb0/6.1.4bf/TriggerObjects/src/TL2D_StorableBank.cc  
        : 1104  TL2D  GlobalLibraryLogger TL2D_StorableBank::get_word()  
         11-Oct-2007 10:52:45 CDT run = 251408 event = 257 
</pre> 
Odds on favorite for blame is Vadim.  Page Level 2 and don't run Trigmon until problem is understood.
<!-- Author: jdl -->
<!-- Comment: Thu Oct 11 12;19;55 comment by...VR -->
actually Vadim is not to blame for this one. Well almost.

This is the mode we've been running since we took the old L2CAL, but you only caught it now. There is no isolist block now, but TrigMon will try to unpack it which results in this error message. TrigMon has to be updated for this.
<!-- Date: Thu Oct 11 12:12:08.532 2007 -->
<table border=0><td><tr><td bgcolor=#e0e030>Do not run TrigMon until further notice or approved by Jonathan Lewis</td></tr></td></table>
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251411' target=_top>251411</a>) -->



<!-- Date: Thu Oct 11 12:19:55.866 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251411>251411</A>
 Terminated at 2007.10.11 12:19:46
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 12:20:57.694 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251411>251411</A>
TERMINATE: COT guys are done. Ilja will pass by now to make some other tests
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 12:21:10.920 2007 -->
Bob and Aseet came back up - they need to do some thinking => will stop cosmics and page CSL so that they can do a 10 min test
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251411' target=_top>251411</a>) -->

<!-- Date: Thu Oct 11 12:27:16.971 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251415>251415</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: TEST_CSL_TORTURE [2,960,651] Ilija is going to test CSL
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 12:30:28.044 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251416>251416</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: TEST_CSL_TORTURE [2,960,651]
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 12:44:15.686 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251417>251417</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: TEST_CSL_TORTURE [2,960,651] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 12:58:03.423 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251418>251418</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: TEST_CSL_TORTURE [2,960,651] Ilija again
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 13:05:33.623 2007 -->
Paged SVT and Massimo came to reboot all SVT crates.
<!-- Author: Manfred -->


<!-- Date: Thu Oct 11 13:09:08.985 2007 -->
I tested the sending of error messages from the CSL monitoring code, which now checks for possible failures and comunicates with the error handler. Carolina was of great help. Thanks!
<!-- Author: Ilija Bizjak -->


<!-- Date: Thu Oct 11 13:12:26.378 2007 -->
RACKS alarm. They are working on Gamma HV, so don't panic.
<!-- Author: carolina -->


<!-- Date: Thu Oct 11 13:13:24.063 2007 -->
Got GAM alarm - WHA HV. At the same time Paolo comes into control room and tells us that they are working on wall HV; alarm will go on and off but we should not panic
<!-- Author: Manfred -->


<!-- Date: Thu Oct 11 13:27:20.779 2007 -->
I've released the WCAL crates.  From what I see now, the problem 
that we were experiencing last week has gone away.   Therefore, 
we don't have plans to swap any hardware on the detector at 
this point.  Some the ranges of valid parameter values may need 
to be adjusted in the TAC consumer before we start routine data 
taking.
<!-- Author: M Jones -->

<!-- Date: Thu Oct 11 13:30:37.662 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251420>251420</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,952,633] Just to test SVT crates which were rebooted (Massimo).
<!-- Author: Carolina Deluca -->
<!-- Comment: Thu Oct 11 13;32;12 comment by...carolina -->
L2 TO


<!-- Comment: Thu Oct 11 13;37;15 comment by...Manfred -->
Looks like Massimo is done - where is my logbook entry?
<!-- Date: Thu Oct 11 13:34:22.297 2007 -->
Calorimeter work<br> 
We will be using/power on-and-off seven calorimeter crates: ccal01, ccal10, ccal12, pcal04, wcal00, wcal03, and wcal06.
<!-- Author: m mattson, s shalhout, s hahn -->

<!-- Date: Thu Oct 11 13:34:52.899 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251420>251420</A>
TERMINATE: Ended because Mark Mattson asked to use crates: ccal01, ccal10, ccal12, pcal04, wcal00, wcal03 and wcal06 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 13:38:39.691 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251421>251421</A>
 Activated at 2007.10.11 13:38:27
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 13:38:40.968 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251421>251421</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->
<!-- Comment: Thu Oct 11 13;41;50 comment by...carolina -->
<pre>* Crates excluded from partition:
ccal01, ccal10, ccal12
pcal04
wcal00, wcal03, wcal06

* Notice we're using trigger table COSMICS[15,973,639] instead of COSMICS[15,954,639] (Laura's request)

* Ignoring reces and svt</pre>
<!-- Comment: Thu Oct 11 13;42;38 comment by...carolina -->
2 minutes running and we have had 3 L2 Decision TO's
<!-- Comment: Thu Oct 11 13;44;27 comment by...carolina -->
Now it's 5 times
<!-- Comment: Thu Oct 11 13;56;41 comment by...carolina -->
<pre>Having lots of L2 Decision TO's

I followed instructions on ace help and looked into the last log files:

NodeControl: waiting for algo unlock (ACK)
NodeControl: NodeControl::ACK received, lcount = 1
NodeControl: sending to TLD: (23) 1192111690:0:5:1:34699:
NodeControl:: Communicator status = [ Communicator::Connected ] 
NodeControl:: { 9:7:18, 10/11/2007 } End of Run. Expecting comm to be closed by TLD. 
NodeControl:: { 9:7:19, 10/11/2007 } Lost connection to control. Reopening...

We are not getting events right now... it's green but RunControl is stucked. I'll restart it.</pre>
<!-- Date: Thu Oct 11 13:39:31.801 2007 -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>Everybody needs to be out of the collision hall tonight by 9 pm. At that time interlock safety personal will come to collision hall to do rad-safety interlock testing. Nobody can be in collison hall at that time.</pre></td></tr></td></table>
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251421' target=_top>251421</a>) -->





<!-- Date: Thu Oct 11 13:44:59.214 2007 -->
We let L2 people (Laura) take over now (finally)
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251421' target=_top>251421</a>) -->



<!-- Date: Thu Oct 11 14:04:19.264 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251422>251422</A>
 Activated at 2007.10.11 14:03:34
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 14:04:20.123 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251422>251422</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] same configuration as before
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 14:09:41.050 2007 -->
<pre>Every time we reach event #16, #32, #48, #64 ... we got a L2 TO... I pressed button TimeOuts in b0control20 but it didn't gave any further information. Log files don't show anything although it could be I'm not looking at the right place. 
Maybe L2 experts should take a look at this. We are ignoring reces, svt and stereo, and L2 is preventing us to run. 
 
 
 
 
 
 
 
 
</pre>
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251422' target=_top>251422</a>) -->
<!-- Comment: Thu Oct 11 14;11;32 comment by...carolina -->
Laura is aware of that and will come in a minute


<!-- Date: Thu Oct 11 14:14:36.403 2007 -->
HBM went red, we are following instructions of the paper on vnode1 but it is still red. For how long do we have to wait until it's green again?
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251422' target=_top>251422</a>) -->
<!-- Comment: Thu Oct 11 14;17;13 comment by...carolina -->
green again
<!-- Date: Thu Oct 11 14:16:09.777 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251422>251422</A>
 Terminated at 2007.10.11 14:15:38
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 14:19:20.054 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251423>251423</A>
 Activated at 2007.10.11 14:18:28
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 14:24:08.989 2007 -->
Technicians surveying water levels returned their keys
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251423' target=_top>251423</a>) -->


<!-- Date: Thu Oct 11 14:30:28.297 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251423>251423</A>
 Terminated at 2007.10.11 14:30:10
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 14:32:22.117 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251424>251424</A>
 Activated at 2007.10.11 14:32:09
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 14:32:56.630 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251424>251424</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 14:33:22.181 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251424>251424</A>
 Terminated at 2007.10.11 14:32:54
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 14:41:58.923 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251425>251425</A>
 Activated at 2007.10.11 14:41:45
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 14:42:23.385 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251425>251425</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 14:42:38.191 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251425>251425</A>
 Terminated at 2007.10.11 14:42:28
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 14:45:02.553 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251426>251426</A>
 Activated at 2007.10.11 14:44:38
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 14:45:03.255 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251426>251426</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 14:45:04.598 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251426>251426</A>
 Terminated at 2007.10.11 14:44:57
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:01:20.505 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251427>251427</A>
 Activated at 2007.10.11 15:00:34
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:01:21.675 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251427>251427</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 15:02:06.868 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251427>251427</A>
 Terminated at 2007.10.11 15:01:57
<!-- Author: RunControl -->



<!-- Date: Thu Oct 11 15:06:40.500 2007 -->
<pre>RACKS alarm again because of Gamma HV WHA3. Jonathan silenced it from iFix since we knew there was people working on that. 
 
Laura is working to fix problem with L2 TO's</pre>
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251427' target=_top>251427</a>) -->
<!-- Comment: Thu Oct 11 15;19;24 comment by...laura -->
done
<!-- Date: Thu Oct 11 15:08:42.836 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251428>251428</A>
 Activated at 2007.10.11 15:07:48
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:08:43.765 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251428>251428</A>
 Terminated at 2007.10.11 15:08:32
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:12:48.535 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251429>251429</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,973,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 15:12:49.530 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251429>251429</A>
 Activated at 2007.10.11 15:10:37
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:12:50.996 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251429>251429</A>
 Terminated at 2007.10.11 15:11:25
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:14:18.356 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251430>251430</A>
 Activated at 2007.10.11 15:13:20
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:14:19.522 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251430>251430</A>
 Terminated at 2007.10.11 15:13:49
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:15:33.639 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251431>251431</A>
ACTIVATE: RunSet: WHA_MANUAL_SOURCING sources moving
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 15:16:30.886 2007 -->
Dale from CMX called, they will trip CMS HV which we can ignore.
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251431' target=_top>251431</a>) -->
<!-- Comment: Thu Oct 11 19;54;06 comment by...Dale and John -->
This occured later then expected <a href=
'/cgi/elog/elog.pl?nb=2007&action=view&page=791&frame=2&anchor=19:03:54.693'>as noted here</a>.
<!-- Date: Thu Oct 11 15:19:11.275 2007 -->
BoB wagner came by; he and Aseet will continue to work on COT and might also set off alarms
<!-- Author: Manfred :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251431' target=_top>251431</a>) -->


<!-- Date: Thu Oct 11 15:23:56.519 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251431>251431</A>
TERMINATE: finished

<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 15:29:24.051 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251433>251433</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,975,639] 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 15:34:35.069 2007 -->
<pre>COT went red on iFix and Power button is red but nothing appears to be out of range... Also the alarm from Process System Alarms pannel is silenced so we guess there is somebody working on there? 
 
Also had a solenoid alarm but scico confirmed they were working there. Cryo alarm, but went red in a few seconds.</pre>
<!-- Author: carolina :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251433' target=_top>251433</a>) -->

<!-- Date: Thu Oct 11 15:37:09.516 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251434>251434</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,975,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 15:44:32.483 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251435>251435</A>
 Activated at 2007.10.11 15:43:45
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:44:33.241 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251435>251435</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,976,639] 
<!-- Author: Carolina Deluca -->


<!-- Date: Thu Oct 11 15:44:55.515 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251435>251435</A>
 Terminated at 2007.10.11 15:44:41
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:47:10.969 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251436>251436</A>
 Activated at 2007.10.11 15:46:35
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 15:55:08.491 2007 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>RunNumber</TH>
<TH>DataType</TH>
<TH>PhysicsTable</TH>
<TH>Begin</TH>
<TH>End</TH>
<TH>LiveTime</TH>
<TH>L3Accepts</TH>
<TH>DelivLumi nb<SUP>-1</SUP></TH>
<TH>LiveLumi nb<SUP>-1</SUP></TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>
<TR><TD>Totals</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD>
<TD>15:55:00</TD>
<TD ALIGN=CENTER>::</TD>
<TD ALIGN=RIGHT></TD>               <TD ALIGN=RIGHT></TD>
<TD ALIGN=RIGHT></TD>
<TD> </TD>
<TD> </TD>
<TD> </TD></TR>

</TABLE>
<!-- Author: End of Shift Report -->


<!-- Date: Thu Oct 11 16:10:50.136 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251436>251436</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,976,639] 
<!-- Author: Carolina Deluca -->



<!-- Date: Thu Oct 11 16:13:03.911 2007 -->
<font color=red>Shift Summary: </font><pre>hift summary: 
 
- still a lot happening in and around the control room 
- current work: 
  . L2/XFT (Laura & Alison) 
  . solenoid up/down (Rob Roser) 
  . COT in pit & upstairs (Aseet & Bob) 
  . WHA (Photos & Paolo) 
  . unknown other work 
- BMU HV is up, other muons too, see details above  
- Do not run TrigMon until further notice 
- Everybody needs to be out of the collision hall 
  tonight by 9 pm for interlock safety personel will come to do rad-safety 
  interlock testing. 
- shift ended with no ACE on duty (ACE in ACE training) plus a fire alarm </pre> 
<!-- Author: Manfred -->

<!-- Date: Thu Oct 11 16:13:07.195 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251436>251436</A>
 Terminated at 2007.10.11 16:13:02
<!-- Author: RunControl -->


<!-- Date: Thu Oct 11 16:29:07.741 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=251437>251437</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: TEST_CSL_TORTURE [2,960,651] CSL test
<!-- Author: Carolina Deluca -->