<!-- Title: Owl shift. Thu Sep 27, 2001 -->
<!-- SciCo: Bob Wagner (ANL) -->
<!-- DAQAce: Boris Iyutin -->
<!-- MonAce: Juan Pablo Fernandez -->
<!-- CO: Fujii Yoshiaki -->
<!-- OpManager: Dave Ambrose -->
<!-- Notes: -->
Shift begins with Tev studies in progress.<br> 
We have a problem with the lift station pump near FCC that has caused sewage backup into first floor counting room and first floor restroom.  Duty mechanic and possibly an electrician are working on the problem.  Called Dispatch (3414) to get an update on the problem.  They spoke with Duty Mechanic via radio.  He asked to get back to them and we were instructed to call after 00:30 if we wished to get an update.


<!-- Date: Thu Sep 27 00:22:33 2001 -->
High voltage remains off to all systems.
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 00:37:20 2001 -->
Pulled floor tile just inside entrance to first floor counting room.  It appears the water level around the drain has gone down a little.  First floor restroom drain also appears to have drained.  Perhaps lift station back on.  Will call Dispatch shortly to try to get information.
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 00:45:06 2001 -->
Called Dispatch.  They inform me that lift station pump is running on a generator.  So we should see the situation improve.  Will page Dave Ambrose to discuss recovery.  
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 00:54:00 2001 -->
Spoke with Dave Ambrose concerning recovery.  He advocates taking it slow with bringing systems on and NOT doing anything until it has dried out.  Then we will assess system by system what we can reactivate.  I will next see about discussing this with the building manager about drying things out.
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 01:16:28 2001 -->
Made an inspection of the subfloor area on first floor.  The drain just inside the counting room door has cleared completely of water.  No cables are submerged at this time but cables and floor are wet in RR 1-9.  Beyond row 9 appears to have not been affected.  We leave floor tiles in place as there is air flow under them and that may dry things out.  For the present, we leave all high voltage systems off.
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 01:37:21 2001 -->
<table border=0><td><tr><td bgcolor=#e0e030>For safety I went ahead and powered off the Gamma supplies for the CSX and CSP.  There are four Gamma supplies (one on WHA, two on CEM, and one on miniskirt) for which the AC power switch has no effect.  For these, I toggled the Local/Remote switch to Remote.  This has the effect of at least turning off the HV to the cable.  Probably should have these supplies repaired as in an emergency, the shift crew might not know how to get them off.  Of course, pulling the AC plug would work.</td></tr></td></table>
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 01:47:54 2001 -->
<table border=0><td><tr><td bgcolor=#e0e030>Koji Terashi ask that we page BSC pager when we are ready to turn on CLC, BSC high voltage.</td></tr></td></table>
<!-- Author: Bob Wagner -->


<!-- Date: Thu Sep 27 02:02:07 2001 -->
<pre>PEM, PHA, PSH are in a dark grey status on the 
"high voltage summary" display (dark grey means 
local PC offline on purpose). I will try to 
restart the client on the 3rd floor plug-PC. 
Done (found the HV monitor stopped and  
restarted it again). 
 
Warnings present in CEM, WHA and intermitently 
in GAM (yellow color) due to those four Gamma supplies 
(one on WHA, two on CEM, and one on miniskirt) for which  
the AC power switch has no effect ( Bob toggled the 
Local/Remote switch to Remote, just to remember).</pre>
<!-- Author: Juan Pablo -->


<!-- Date: Thu Sep 27 02:05:42 2001 -->
Successfully tested the L2 error handling code which will 
halt the run after detecting errors in the data arriving 
at L2. This will allow us to run with cutting at L2. 
Will do a test of cutting on Jets at L2 when there is beam.
<!-- Author: Stephen, Peter -->


<!-- Date: Thu Sep 27 02:19:37 2001 -->
<pre>TOF is missing the heartbeat. I will 
proceed as taught by Stephano (TOF expert)... 
TOF HV monitor program back up. 
 
CEM,CHA,WHA on a light pink state (local PC 
unaccesible). Konstantin Kotelnikov (CAL expert)  
is on the 1st floor (we should ignore them by now). 
 
</pre>
<!-- Author: Juan Pablo -->
<!-- Comment: Thu Sep 27 02;32;49 comment by...Juan Pablo -->
<pre>comunication with CDF_PISA_BOX PC interrupted
by Konstantin (PMT HV and GAMMA processes are not
running). He asked us to leave things in such a
state till we bring everything-all the HV- up)</pre>


<!-- Date: Thu Sep 27 02:36:19 2001 -->
<pre>PEM alarm . It cleared itself in less than 1 minute. 
I could see anyway which crate was the alarm  
comming from (crate 1 on the west plug).  
Not in a alarm state anymore (will keep 
an eye open). </pre>
<!-- Author: Juan Pablo -->


<!-- Date: Thu Sep 27 03:21:10 2001 -->
writing my electron validation code, praying for the month long shut down to come so I can sleep
<!-- Author: hray -->


<!-- Date: Thu Sep 27 04:12:57 2001 -->
Checked the progress of drying out on first floor.  Area around drain is dry.  The cables under row 6 of racks still wet, but appear to be somewhat drier.<br>
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 04:34:20 2001 -->
<pre>Another of this alarms from PEM that goes away 
Crate 01 EM : Gr "east" CAEN err 
looked at history (by selecting "PEM -&gt; Diagnostic -&gt;show historical data" and graph showed nothing last hour)</pre>
<!-- Author: Juan Pablo -->


<!-- Date: Thu Sep 27 06:49:47 2001 -->
MCR calls.  They'd like to see LostP, etc. for loss studies.  Told them can't do until we can have cables verified safe for HV operation.
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 07:01:29 2001 -->
Page BSC to try to get BSC and CLC on so as we can give data to MCR for loss studies.  Juan Pablo will restore the voltage with help from Koji over the phone.
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Thu Sep 27 07:06:26 2001 -->
LostP, etc. are counting once more as CLC and BSC are back on.  
<!-- Author: Bob Wagner (ANL) -->
<!-- Comment: Thu Sep 27 07;12;12 comment by...RGW -->
In speaking with MCR to let them know the BSC is coming back on, we query about access.  It will be at 08:30 for 1 hour.
<!-- Date: Thu Sep 27 07:11:35 2001 -->
<pre>CLC,BSC and RPS HVs are ON as 
requested by MCR 
no problems arose in the procedure 
(done with the supervise of Koji 
by phone)</pre>
<!-- Author: Juan Pablo  -->



<!-- Date: Thu Sep 27 07:54:47 2001 -->
<font color=red>Shift Summary: </font><br><font size=+1><strong>Owl Shift Summary by SciCo</strong></font><br> 
<p> 
00:00	Problem continues with the sewage backup in first floor counting room. 
	High voltage remains off for all systems.  Asked Dispatch for update. 
	They ask that we call back after about 00:30. 
</p><p> 
00:45	Lift station pump running on a generator.  Spoke with Dave Ambrose about 
	recovery.  He suggests to go slow.  Let things dry out. 
</p><p> 
01:15	Drain just inside door of first floor counting room is clear of water. 
	No submerged cables but floor and cables are still wet.  Leave HV off. 
</p><p> 
01:37	For safety, powered off remaining Gamma supplies that were still on. 
	Four have ineffective AC switch so toggled them to Remote control which 
	shuts off HV since they have no remove input. 
</p><p> 
01:47	Asked to call BSC pager when time to turn on CLC and BSC. 
</p><p> 
02:02	Juan Pablo restarted plug client on 3rd floor.  Also got TOF HV monitor 
	program going again to restore heartbeat. 
</p><p> 
02:00	Since not much to do, glue Nixon back together. 
</p><p> 
06:50	MCR calls to ask about LostP, etc.  They want to do studies and need to 
	see the counts.  Explain the situation to them. 
</p><p> 
07:00	After discussing with Dave Ambrose, we decide to page Koji and get BSC 
	back on for MCR's use.  Done. 
</p>   
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   zip
Delivered Luminosity     
Acquired Luminosity      
Efficiency             
   
</td></tr></table></pre>
<!-- Author: Bob Wagner -->


<!-- Date: Thu Sep 27 08:00:47 2001 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2001&action=view&page=-2447&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2001&action=view&page=-2447  width='300' align='top'></td><td>svx rad losses during this owls shift numbers: SVRAD0 : .421 SVRAD2 : .9572 SVRAD3 : 2.7472</a></td></tr></table>
<!-- Author: Juan Pablo  -->


<!-- Date: Thu Sep 27 08:04:51 2001 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2001&action=view&page=-2448&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2001&action=view&page=-2448  width='300' align='top'></td><td>p pb losses</a></td></tr></table>
<!-- Author: Juan Pablo -->