<!-- Title: Eve shift. Sat Aug  2, 2003 -->
<!-- SciCo: Steve Kuhlmann -->
<!-- DAQAce: Paola Squillacioti -->
<!-- MonAce: Dan Ryan -->
<!-- CO: Dmitri Smirnov -->
<!-- OpManager: Kaori/Andy -->
<!-- Notes: -->
<pre>No beam, finishing work on TEL.   
Tev restart within an hour.  Hopefully a  
shot tonight. </pre>

<!-- Date: Sat Aug 2 16:22:03.303 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166972">166972</A>
 Activated at 2003.08.02 16:18:44
<!-- Author: RunControl -->



<!-- Date: Sat Aug 2 16:22:03.205 2003 -->
<pre>Stack is at 203.</pre>
<!-- Author: Steve Kuhlmann -->

<!-- Date: Sat Aug 2 16:22:25.951 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166972">166972</A>
ACTIVATE: Restart a cosmic_silicon_tape
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 16:25:01.770 2003 -->
During the partitioning I had a Procmon error: the problem was with CSL MAIN, I stopped it and restarted.
<!-- Author: Paola -->


<!-- Date: Sat Aug 2 16:28:07.042 2003 -->
b0dap63.fnal.gov:ConsumerErrorRe:4:23:10 PM->Runtime Error 1, Event 5, RunNum 166972: SvxMon Halt Recover Run: Pipeline out of synch in 119 silicon readout chips.
<!-- Author: Paola :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166972' target=_top>166972</a>) -->


<!-- Date: Sat Aug 2 16:38:18.746 2003 -->
<pre>Cryo calls.  Recent very brief gas alarm is due to  
very sensitive sensor in ArCo2 system.  He is 
going to the trailer to investigate,  will call 
when he returns. </pre>
<!-- Author: Steve Kuhlmann -->
<!-- Comment: Sat Aug  2 17;18;47 comment by...SK -->
<pre>Upon return,  Jim says gas level in that trailer
was low and he switched the ArCo2 to the
other trailer.  </pre>
<!-- Date: Sat Aug 2 16:41:32.995 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166972">166972</A>
 Terminated at 2003.08.02 16:37:36
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 16:41:34.328 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166972">166972</A>
TERMINATE: end run to restart csl
<!-- Author: Paola 2080 -->
<!-- Comment: Sat Aug  2 17;15;49 comment by...Paola -->
The Co couldn't restart ymon, so we tried to stop and restart csl again, and now everithing is ok.
<!-- Date: Sat Aug 2 16:51:54.846 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166974">166974</A>
 Activated at 2003.08.02 16:47:29
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 16:51:55.378 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166974">166974</A>
ACTIVATE: cosmic_silicon_tape
<!-- Author: Paola 2080 -->




<!-- Date: Sat Aug 2 17:16:24.041 2003 -->
IMON was dead (connection to b0dau30 closed by foreign host).  Restarted.
<!-- Author: dan :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166974' target=_top>166974</a>) -->



<!-- Date: Sat Aug 2 18:05:23.465 2003 -->
CMX HV trip, SE 16 and 17.  Reset them.
<!-- Author: dan :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166974' target=_top>166974</a>) -->
<!-- Comment: Sat Aug  2 18;08;38 comment by...Paola -->
Trigger inhibit for CMU Central Muon (CMU, CMX, CMP)->Paused the run-> Resumed the run after Dan recovered the trip

<!-- Date: Sat Aug 2 18:19:08.703 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166974">166974</A>
 Terminated at 2003.08.02 18:15:25
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 18:19:09.768 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166974">166974</A>
TERMINATE: end cosmic
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 18:26:47.825 2003 -->
<pre>Waiting.  Cosmics.  Stack at 209 and growing.</pre>
<!-- Author: SK -->

<!-- Date: Sat Aug 2 18:32:39.048 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166978">166978</A>
 Activated at 2003.08.02 18:28:46
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 18:32:40.562 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166978">166978</A>
ACTIVATE: cosmic_silicon_tape 
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 18:34:37.769 2003 -->
We swapped in the second Pulsar Rx board
<!-- Author: Reisert :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166978' target=_top>166978</a>) -->
<!-- Comment: Sat Aug  2 19;24;45 comment by...SK -->
<pre>Based on discussion with Kaori we will add 
the Pulsar board to the default physics
configurations.  </pre>


<!-- Comment: Sat Aug  2 20;20;55 comment by...SK -->
<pre>Ok, Pulsar crate not board.</pre>
<!-- Date: Sat Aug 2 19:27:32.315 2003 -->
<pre>Waiting.  Cosmics.  Stack at 210. 
MCR log says they will take out 
all but 30ma and "load high" on the protons.</pre>
<!-- Author: SK -->

<!-- Date: Sat Aug 2 20:19:38.070 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166978">166978</A>
 Terminated at 2003.08.02 20:16:06
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 20:20:03.076 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166978">166978</A>
TERMINATE: end cosmic_silicon_tape to put the HV at standby
<!-- Author: Paola 2080 -->




<!-- Date: Sat Aug 2 20:26:14.556 2003 -->
HV at standby.
<!-- Author: dan -->

<!-- Date: Sat Aug 2 20:33:23.901 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166979">166979</A>
 Activated at 2003.08.02 20:28:27
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 20:36:07.378 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166979">166979</A>
ACTIVATE: cosmic run
<!-- Author: Paola 2080 -->


<!-- Date: Sat Aug 2 20:45:15.188 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166979">166979</A>
 Terminated at 2003.08.02 20:41:38
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 20:45:16.039 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166979">166979</A>
TERMINATE: end run
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 20:48:03.310 2003 -->
CMX HV trip on NW 2 and 3.  3 failed attempts to bring back to standby (on 3rd attempt tripped at V=1700, I=5.7).  Waited 10 minutes before 4th attempt.  Success!
<!-- Author: dan/paola -->
<!-- Comment: Sat Aug  2 21;17;53 comment by...dan -->
Another CMX HV trip in Standby on NW 2&3.  This time waited ~5min before bringing up voltages and it worked the first time.
<!-- Comment: Sat Aug  2 22;01;33 comment by...dan -->
3rd CMX HV trip in 20 minutes.  This time NW 0-3.  Paged expert and he suggested bringing voltage ON and then back to STANDBY.  We did this (the entire CMX) and all was fine for 30min.  In the meantime, Sal called back to suggest that humidity was to blame.  If during data taking this wedge trips, to just turn it off and mark run questionable.
<!-- Comment: Sat Aug  2 22;05;56 comment by...dan -->
4th CMX trip on NW 2&3.  30 seconds later, trip on SW 7&8 and 9&10. Outer 7&8 is purple (communication error), so we are beginning to suspect a software problem.  Phil is on the phone with the sci-co.
<!-- Comment: Sun Aug  3 19;41;02 comment by...Phil -->
<p>I was too eager to take off my hip boots and get back in my jammies to put in an entry last night about the CMX trips. (Steve summarized things ok.) The trips were from standby. The grids seemed to be tripping as there were no anode only trips. Most were on one HV channel (2/3 W) but others on the top W side tripped as well. None of the others went more than once, but they spread across the entire top west side.</p>
<p>None of this seemed consistent with a detector problem.</p>
<p>One thing I should have done and didn't was to take the camera and look for water.</p>
<!-- Comment: Wed Aug 13 11;04;12 comment by...ps -->
A belated remark: I plotted the icicle data from the humidity sensors and saw nothing anomalous. I looked at the same data for the chamber currents and saw nothing of iterest (Fat chance, the data is too sparse to be of much use. But I looked anyway.)
<!-- Date: Sat Aug 2 21:14:45.483 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166981">166981</A>
ACTIVATE: AAA_shotsetup with physics_test_1_05[20,325,397]
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 21:17:52.093 2003 -->
Fatal clock error, there were two errors: a timing error and a seq_hold error. We cleared the first one pushing the usual clear errors, for the second one we paged Steve that said we can try to reset the error pushing the clear errors on the right below the seq_hold. The error cleared and now we are running the  AAA_shotsetup
<!-- Author: Paola -->
<!-- Comment: Sat Aug  2 21;19;57 comment by...Paola -->
If it happens again during the data taking we must clear the error and do an HRR, if happen again clear the error and restart a new run 

<!-- Date: Sat Aug 2 21:22:00.037 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166981">166981</A>
TERMINATE: end AAA_shotsetup 
<!-- Author: Paola 2080 -->


<!-- Date: Sat Aug 2 21:29:53.071 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166982">166982</A>
ACTIVATE: AAA_shotsetup with default trigger table
<!-- Author: Paola 2080 -->
<!-- Comment: Sat Aug  2 21;54;34 comment by...Paola -->
Aborted the run
<!-- Date: Sat Aug 2 21:41:29.077 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166982">166982</A>
Before arriving in the activate state we had RXPT Error LOOP !!!  
5 crate/s: b0l1cl00(128), b0l1cl01(129), b0l1cl02(130), b0l1cl03(131), b0l1gl00(134),  in error.[RXPT] 
In the VxWorks System monitor the crates b0pcal01, b0pcal02, b0pcal05, b0pcal07 and b0pcal11 are red, so I rebooted them. 
Now I'm in the start state for silicon HV problem.  
<!-- Author: Paola -->


<!-- Date: Sat Aug 2 21:59:18.553 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166983">166983</A>
ACTIVATE: AAA_shotsetup with default trigger table
<!-- Author: Paola 2080 -->


<!-- Date: Sat Aug 2 22:01:30.505 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166983">166983</A>
TERMINATE: end AAA_shotsetup
<!-- Author: Paola 2080 -->




<!-- Date: Sat Aug 2 22:11:07.393 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166984">166984</A>
ACTIVATE: L2_torture
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 22:14:27.047 2003 -->
<pre>SVX HV trips (no beam in the machine when tripped) on:   
 
B0 W8,5,4,0 
B1 W8 
B2 W8,5,4,3 
B3 W8,11 
B4 W9,5,4,3,2 
B5 W7,6,5,4,0 
 
Paged expert, Reed OK's the resetting of wedges.  While reseting, another trip on SVX ladder B4 W9 L4.  Paged Reed again, says this is okay, to leave tripped until he arrives.</pre>
<!-- Author: dan -->


<!-- Date: Sat Aug 2 22:45:13.568 2003 -->
HBM red.  Cleared in minutes.
<!-- Author: dan :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166984' target=_top>166984</a>) -->


<!-- Date: Sat Aug 2 22:45:48.861 2003 -->
<pre>MCR says shot setup in 10 minutes.  Stack 218ma.</pre>
<!-- Author: SK -->


<!-- Date: Sat Aug 2 22:56:14.016 2003 -->
Firus message at 22:25 "TROUBLE:3  All Mini's have been rebooted and will re-report alarms."  
<!-- Author: dan :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166984' target=_top>166984</a>) -->
<!-- Comment: Sat Aug  2 23;16;46 comment by...dan -->
<pre>The above was for Red Front End.  But now more messages:

TROUBLE
22:54 Red Front End
22:52 Backup Database

And now:  23:08 Blue Front End--Rebooting all Mini's</pre>

<!-- Date: Sat Aug 2 23:36:48.176 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166984">166984</A>
TERMINATE: end L2_torture
<!-- Author: Paola 2080 -->


<!-- Date: Sat Aug 2 23:41:12.549 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166985">166985</A>
 Activated at 2003.08.02 23:37:53
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 23:41:30.747 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166985">166985</A>
ACTIVATE: ace_cosmic_all
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 23:43:28.633 2003 -->
<pre>Final protons.  </pre>
<!-- Author: SK -->


<!-- Date: Sat Aug 2 23:44:07.638 2003 -->
 
 
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2003.08.02</td><td>23:34:12</td><td>W Inner BLM</td><td>0.08</td><td>RADS</td></tr> 
<tr><td>2003.08.02</td><td>23:34:12</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.08.02</td><td>23:34:12</td><td>E Inner BLM</td><td>0.17</td><td>RADS</td></tr> 
<tr><td>2003.08.02</td><td>23:34:12</td><td>E Outer BLM</td><td>0.54</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong>
<!-- Author: dan ryan :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166985' target=_top>166985</a>) -->

<!-- Date: Sat Aug 2 23:48:21.394 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166985">166985</A>
 Terminated at 2003.08.02 23:44:28
<!-- Author: RunControl -->


<!-- Date: Sat Aug 2 23:48:22.446 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166985">166985</A>
TERMINATE: end ace_cosmic_all to start AAA_shotsetup
<!-- Author: Paola 2080 -->


<!-- Date: Sat Aug 2 23:54:58.600 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=166986">166986</A>
ACTIVATE: AAA_shotsetup
<!-- Author: Paola 2080 -->



<!-- Date: Sat Aug 2 23:56:38.643 2003 -->
<font color=red>Shift Summary: </font><pre>16:00 Start with no beam after TEL work.  Stack at 203. Cosmics. 
20:26 Hv at standby.  Beam going into TeV. 
21:00 Series of CMX trips from standby, top west side.  
Some are certainly not beam related. Talking to experts. 
21:14 Kaori comes in to watch the test trigger table which 
drops the cot and svx Dbanks.  Test looks ok she says. 
21:20 SVX trips.  Reed comes in and resets.  
21:30 Considerable lightning.  Aborts in MI and TeV during tuneup.  
22:11 More CMX trips.  Phil comes in.  Says it's most likely humidity. 
He says if it happens during the run the only choice is to turn off  
the offending section for a while and mark the run bad for muons.   
(And try to reset periodically) 
22:52 Shot setup begins. 
23:43 Final protons being loaded</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   
Delivered Luminosity   0  
Acquired Luminosity    0  
Efficiency             100%
   
</td></tr></table></pre>
<!-- Author: Steve Kuhlmann -->

<!-- Date: Sat Aug 2 23:58:55.186 2003 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>23:55:02</TD>                                                       
<TD ALIGN=CENTER>::</TD>                                                
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->