<!-- Title: Owl shift. Mon Apr 15, 2002 -->
<!-- SciCo: Avi YAgil -->
<!-- DAQAce: Nicolo Moggi -->
<!-- MonAce: Phil Yoon -->
<!-- CO: Andrew Ivanov -->
<!-- OpManager: Bill -->
<!-- Notes: -->
<pre>Store 1207 in progress. Lumi 7E30. 
Not taking data. COT SL1,2 trip. Aseet on his way. 
</pre>

<!-- Date: Mon Apr 15 00:27:08 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143284">143284</A>
 Terminated at 2002.04.15 00:25:43
<!-- Author: RunControl -->



<!-- Date: Mon Apr 15 00:32:59 2002 -->
COT SL1,2 trip will not be understood before the next access. Could be broken wire. As there's risk involved in trying to run the chamber, we end the run, and will restart without the COT. Aseet investigating.
<!-- Author: Avi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143284' target=_top>143284</a>) -->


<!-- Date: Mon Apr 15 00:37:50 2002 -->
<pre> 
Current CDF Roman Pot postions: 
  <li> TOPT1M: -15.8 MILS </li>  
  <li> TPOT2M:  -6.2 MILS </li> 
  <li> TPOT3M: +10.0 MILS </li> 
</pre>  
<!-- Author: py -->

<!-- Date: Mon Apr 15 00:39:07 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143285">143285</A>
 Activated at 2002.04.15 00:37:47
<!-- Author: RunControl -->



<!-- Date: Mon Apr 15 00:40:03 2002 -->
Begin a new run with no COT crates. XFT crates still in... is this ok ?
<!-- Author: niccolo&#39; :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143285  ' target=_top>143285  </a>) -->


<!-- Date: Mon Apr 15 01:13:24 2002 -->
<pre> 
The white board request Mon Aces to monitor the beam halo variables. 
However, seemingly without any further notice since last Wed.,
I'm not quite certain that the variables are reliable at this moment. 
Anyhow, here's the current showdown: 
 
 <li> B0PHSM: ~ 280 KHz </li> 
 <li> B0AHSM: ~ 18.8 KHz </li> 
</pre> 
 
<!-- Author: py -->


<!-- Date: Mon Apr 15 01:40:51 2002 -->
Trying to find if there was a spike in losses causing the trip.
Called MCR and asked their advice/help in investigating this. 
<!-- Author: avi :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143285' target=_top>143285</a>) -->


<!-- Date: Mon Apr 15 01:40:52 2002 -->
We are aware that, during the previous Eve. shift, the COT trip occurred somewhere between 23:40 - 23:45PM. So, We looked at various variabls for losses: LOSTP, SVRAD, and B0RAT in this  
time window. But, no spikes observed. 
<!-- Author: Avi, py -->
<!-- Comment: Mon Apr 15 02;05;17 comment by...py -->
Please see the 4 plots below.
<!-- Date: Mon Apr 15 02:03:07 2002 -->
<pre>Status on COT problems: 
Short, 100k on all sense, 200k on potential for: 
SL3, Quad A : P12, S11, P11, P10  
SL2, Quad D : S6, S5, P9 
 
There is no current draw at 0.4% with the above disconnected. 
This implies a broken wire in SL3, Quad A near 0 degrees, falling into SL2, Quad D. 
 
Shift crew tried to reset from trip at 23:41 which lost most of 
current history, but a spike was still visible on all channels 
of SL2. (SL3 history was gone.)  
 
Turned off all Superlayers at crowbar, killed HV control/display 
program so it won't get turned on upstairs. 
 
Note: Scope did not trigger; Aseet pressed "force trigger" to 
check voltages.</pre>
<!-- Author: Aseet, Robyn -->


<!-- Date: Mon Apr 15 02:04:13 2002 -->

<center><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2133&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2133' width='300'></a><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2134&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2134' width='300'></a><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2135&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2135' width='300'></a><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2136&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2136' width='300'></a></center>
<!-- Author: py -->
<!-- Comment: Mon Apr 15 02;08;10 comment by...py -->
Beam losses, SVX Rad, B0 Rate Meters, and Beam Halos during a 5min time window around COT trip.


<!-- Date: Mon Apr 15 03:27:06 2002 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2002&action=view&page=-2137&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2002&action=view&page=-2137  width='300' align='top'></td><td>B0LLUM: Isn't this related to the COT trip?</a></td></tr></table>
<!-- Author: Aces -->
<!-- Comment: Mon Apr 15 08;14;20 comment by...Jim Patrick -->
This means the "Live" luminosity dropped to zero
at this time. Which means the trigger inhibit
stopped data taking, like it should.
<!-- Date: Mon Apr 15 03:47:29 2002 -->
<table><tr><td><a href=/elog/notebooks/2002/Mon_Apr_15_2002_Owl.034713.ps align=top><img src=/cgi/elog/elog.pl?nb=2002&action=view&page=-2139  align='top'></td><td>Trigger tower energy plots have two many off-diagonal  entries. Not sure how serious this problem is. Trigger rates look fine.  </a></td></tr></table>
<!-- Author: Andrew -->
<!-- Comment: Mon Apr 15 03;49;01 comment by...Andrew -->
Run 143285.

<!-- Comment: Mon Apr 15 04:02 comment by...Avi -->
Since we do not have COT and Si, this will wait for the morning.

<!-- Date: Mon Apr 15 04:57:42 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143285">143285</A>
 Terminated at 2002.04.15 04:55:33
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 05:06:12 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143286">143286</A>
 Activated at 2002.04.15 05:04:41
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 05:08:27 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143286">143286</A>
 Terminated at 2002.04.15 05:06:18
<!-- Author: RunControl -->



<!-- Date: Mon Apr 15 05:08:28 2002 -->
<pre>Got a DTO from crate b0htdc00. It did not recover with HRR and 
popped up a new orange window saying  
 
HRR forced 
Run PAUSED 
Check b0htdc00 and resume run 
 
The crate was still responding but I reset it anyway and resumed run. This did not solve the problem so I try to end the run.</pre>
<!-- Author: niccolo&#39; :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143285' target=_top>143285</a>) -->


<!-- Date: Mon Apr 15 05:21:32 2002 -->
<pre>We cleaned up EVB and try again.</pre>
<!-- Author: niccolo&#39; -->
<!-- Comment: Mon Apr 15 05;37;38 comment by...niccolo&#39; -->
Did not help !!!
<!-- Comment: Mon Apr 15 08;19;44 comment by...patrick -->
<pre>These are bunch counter mismatch errors, and
are local to the front-end crate.
Cleaning up the event builder will not fix them.

(MLE) b0htdc00:Messenger:4:55:38 AM->Event 3: Bunchcounters in slot 4 and slot 18 disagree (at least)
(MLE) b0htdc00:Messenger:4:55:38 AM->Runtime Error 3, Event 3: Bunch counter mismatch, mismatch count = 1
(MLE) b0htdc00:Messenger:4:55:38 AM->Event 4: Bunchcounters in slot 4 and slot 18 disagree (at least)
(MLE) b0htdc00:Messenger:4:55:38 AM->Runtime Error 4, Event 4: Bunch counter mismatch, mismatch count = 1
</pre>
<!-- Date: Mon Apr 15 05:25:43 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143287">143287</A>
 Activated at 2002.04.15 05:23:09
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 05:25:43 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143287">143287</A>
 Terminated at 2002.04.15 05:24:29
<!-- Author: RunControl -->



<!-- Date: Mon Apr 15 05:43:08 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143288">143288</A>
 Activated at 2002.04.15 05:41:27
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 05:45:23 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143288">143288</A>
 Terminated at 2002.04.15 05:42:24
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 06:25:10 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143291">143291</A>
 Activated at 2002.04.15 06:23:06
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 06:25:37 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143291">143291</A>
 Terminated at 2002.04.15 06:24:13
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 06:27:43 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143291">143291</A>
 Terminated at 2002.04.15 06:24:13
<!-- Author: RunControl -->


<!-- Date: Mon Apr 15 06:38:25 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143292">143292</A>
 Activated at 2002.04.15 06:36:45
<!-- Author: RunControl -->



<!-- Date: Mon Apr 15 06:43:26 2002 -->
<pre>Let me try to write down what is happening here. 
- first we got a DTO from b0htdc00 which did not recover by HRR; 
- we reset the crate and started a new run, but DTO windows         keeped popping up.... 
- we cleaned up EVB: same as above... 
- talked with the expert (L2 general pager). TDC experts do not     respond at all; 
- following suggestion of the expert, we powercycled b0htdc00, 
  started a new run.... and got the same problem again !!!!! 
- used crate_reset but still having DTOs 
- together with SciCo decided to remove the crate out of the 
  run configuration and start a new run (n^th time); 
- this time we got a BUSY Time Out with some instrunctions from 
  RC: 
	End+Reset 
	Cleanup EVB 
	Start new Run 
  We did and are now running with no errors (for the moment!!!) 
 </pre>
<!-- Author: ACEs -->
<!-- Comment: Mon Apr 15 07;03;45 comment by...aces -->
1.5 hrs have already passed, but still no response from the TDC expert after mutiple pagings! 
 


<!-- Comment: Mon Apr 15 09;04;03 comment by...Frank Chlebana -->
<pre>You should recognize that the errors reported are local
to the crate and cleaning up the EVB will not help, so
this step was not needed.

If you are having persistant problems with the crate, 
after trying the crate reset, you can remove the crate
from the run and debug the problem in a seperate partition.
</pre>
<!-- Date: Mon Apr 15 07:45:59 2002 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2002&action=view&page=-2140&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2002&action=view&page=-2141  width='300' align='top'></td><td>Proton (green) and antiproton(cyan) halo during the time the COT tripped.</a></td></tr></table>
<!-- Author: R.J. Tesarek -->


<!-- Date: Mon Apr 15 07:50:28 2002 -->
<center><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2142&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2143' ></a><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2144&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2145' ></a></center>
<!-- Author: R.J. Tesarek -->
<!-- Comment: Mon Apr 15 07;52;50 comment by...R.J. Tesarek -->
<pre>Proton(left) and antiproton(right) halo counter rates.  
Green  = North,   
Cyan   = Top,   
Yellow = South,   
Red    = Bottom   
Colors hold for both plots.  Due to a lack of space in our
data logger, these parameters are logged only every 5 minutes.</pre>

<!-- Date: Mon Apr 15 07:56:18 2002 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143284>143284</A> 
x22FB4</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_01 [4,280]</TD>                                           
<TD ALIGN=CENTER>22:09:59</TD>                                          
<TD ALIGN=CENTER>00:25:43</TD>                                          
<TD ALIGN=CENTER>01:26:51</TD>                                          
<TD ALIGN=RIGHT>     15,100,773</TD>                                    
<TD ALIGN=RIGHT>       728,308</TD>                                     
<TD ALIGN=RIGHT>      77,410</TD>                                       
<TD ALIGN=RIGHT>   40.660</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143285>143285</A> 
x22FB5</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_01 [4,280]</TD>                                           
<TD ALIGN=CENTER>00:37:47</TD>                                          
<TD ALIGN=CENTER>04:55:33</TD>                                          
<TD ALIGN=CENTER>04:07:05</TD>                                          
<TD ALIGN=RIGHT>      4,359,098</TD>                                    
<TD ALIGN=RIGHT>       446,385</TD>                                     
<TD ALIGN=RIGHT>     101,976</TD>                                       
<TD ALIGN=RIGHT>   93.917</TD>                                          
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=143292>143292</A> 
x22FBC</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_01 [4,280]</TD>                                           
<TD ALIGN=CENTER>06:36:45</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>01:17:36</TD>                                          
<TD ALIGN=RIGHT>      1,117,343</TD>                                    
<TD ALIGN=RIGHT>       119,082</TD>                                     
<TD ALIGN=RIGHT>      29,865</TD>                                       
<TD ALIGN=RIGHT>   23.823</TD>                                          
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>07:55:01</TD>                                                       
<TD ALIGN=CENTER>06:51:33</TD>                                          
<TD ALIGN=RIGHT>      20,577,214</TD>                                   
<TD ALIGN=RIGHT>      1,293,775</TD>                                    
<TD ALIGN=RIGHT>       209,251</TD>                                     
<TD ALIGN=RIGHT>  158.400</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Mon Apr 15 08:05:43 2002 -->
<center><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2150&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2150' width='300'></a><a href='/cgi/elog/elog.pl?nb=2002&action=view&page=-2149&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2002&action=view&page=-2149' width='300'></a></center>
<!-- Author: py -->
<!-- Comment: Mon Apr 15 08;05;59 comment by...py -->
Summary Plots



<!-- Date: Mon Apr 15 08:17:44 2002 -->
<font color=red>Shift Summary: </font><pre>	Owl Shift Summary 
	April 15, 2002 
 
COT down with trip in SL1 and 2. Aseet thinks its a broken wire in SL3 around  
phi=0. We are looking at every possible variable in ACNET around the time of  
trip. Aseet and Robyn disabled the COT HV controls, such that it cannot be  
accidentally powered on by a shift crew.  
Taking data without Si and COT. 
 
0340 Miss-match found by TrigMon between DCAS & ADMEM.  
     Noted in Elog. Did not think it worth waking up an expert. 
 
0640 Been unable to run since ~0500. Problem started with a hdtc crate  
     hanging. See aces Elog entry for details. 
     >>> again, it seems that the procedures to deal with this type of  
     (expected) problems are not adequate. 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   143284-143292
Delivered Luminosity   345.9  
Acquired Luminosity    238.1  
Efficiency             68.8
   
</td></tr></table></pre>
<!-- Author: Avi -->