<!-- Title: Owl shift. Mon Mar 12, 2007 -->
<!-- SciCo: Rob Snihur -->
<!-- DAQAce: Jim Lungu -->
<!-- MonAce: &nbsp; -->
<!-- CO: Kazuhiro Yamamoto -->
<!-- OpManager: Sasha Pronko -->
<!-- Notes: -->
<pre>Let's get some data from this store before the access! 
Current lumi = 99. 
</pre>


<!-- Date: Mon Mar 12 00:38:42.662 2007 -->
<pre>Trigger Inhibit is set from the components: 
 VME Power  		[147] VME Crate Power Supplies from iFix 
 
fan status for 1RR21F_1 was not ok, the cryo tech recovered it, and now taking data</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284' target=_top>236284</a>) -->
<!-- Comment: Mon Mar 12 02;38;42 comment by...Jim -->
<pre>from the cryo elog, it looks like the technician didn't do anything because the alarm cleared by itself in 20 sec</pre>
<!-- Date: Mon Mar 12 00:51:23.103 2007 -->
<pre>TevMon alarm: abort gap losses for both protons and pbars went up. RFsum and RFsumA are around 0.8 as well. The AGIGi2 went up too. I halted the run, put silicon on stdby, paged the silicon.</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284' target=_top>236284</a>) -->
<!-- Comment: Mon Mar 12 00;52;53 comment by...Jim -->
<pre>from MCR log:

TRF4 has tripped off; lost some lum as well. DZero lum HV tripped too?

The pbar abort gap losses are high and not stable</pre>


<!-- Date: Mon Mar 12 00:54:03.396 2007 -->
<center><a href='/cgi/elog/elog.pl?nb=2007&action=view&page=-2689&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2007&action=view&page=-2689' width='300'></a><a href='/cgi/elog/elog.pl?nb=2007&action=view&page=-2690&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2007&action=view&page=-2690' width='300'></a></center>
<!-- Author: Kazuhiro -->


<!-- Date: Mon Mar 12 00:56:01.356 2007 -->
<table><tr><td><a href=/elog/notebooks/2007/Mon_Mar_12_2007_Owl.005536.ps align=top><img src=/cgi/elog/elog.pl?nb=2007&action=view&page=-2692  align='top'></a></td><td>MET45 had something like a spike at the beginning of the run. But it is fixed now.</td></tr></table>
<!-- Author: Kazuhiro -->

<!-- Date: Mon Mar 12 00:58:00.851 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284>236284</A>
fwd: <tt>West IN/OUT MP Occupancy empty.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->


<!-- Date: Mon Mar 12 00:58:01.753 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284>236284</A>
general: <tt>CO checklist done. No new problems.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->



<!-- Date: Mon Mar 12 01:01:21.909 2007 -->
<center><a href='/cgi/elog/elog.pl?nb=2007&action=view&page=-2693&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2007&action=view&page=-2693' width='300'></a><a href='/cgi/elog/elog.pl?nb=2007&action=view&page=-2694&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2007&action=view&page=-2694' width='300'></a></center>
<!-- Author: Jim -->
<!-- Comment: Mon Mar 12 01;01;45 comment by...Jim -->
<pre>the losses during the RF trip</pre>

<!-- Date: Mon Mar 12 01:04:23.200 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284>236284</A>
 Terminated at 2007.03.12 01:02:55
<!-- Author: RunControl -->


<!-- Date: Mon Mar 12 01:04:24.116 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284>236284</A>
TERMINATE: TevMon still in alarm (CDF permit/inhibit) and the silicon guys didn't respond; we'll try again, until then run nosilicon
<!-- Author: Jim -->
<!-- Comment: Mon Mar 12 01;11;45 comment by...Jim -->
<pre>got this while ending the run:

Attention!!!. L3_RELAY_FAILURE Error !!!. 

 One or more L3 nodes is likely to have died.
 Do not panic - this message is mainly for your information.
 Level3 should be able to recover by itself.
 Please read the error log for more specific instructions.

manually overriden transition to Idle, then in start cleaned-up L3 farm, no starting the nosilicon run</pre>

<!-- Comment: Mon Mar 12 01;55;40 comment by...Ignacio -->
I am sorry for the slow response, but I only have got the second page. Please call me to any of the two phone numbers hang on the board if the response is too slow. 
<!-- Date: Mon Mar 12 01:13:01.459 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236285>236285</A>
 Activated at 2007.03.12 01:12:53
<!-- Author: RunControl -->


<!-- Date: Mon Mar 12 01:13:02.385 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236285>236285</A>
ACTIVATE: RunSet: AAA_NOSILICON  Trigger: PHYSICS_4_01 [5,866,678] 
<!-- Author: Jim -->


<!-- Date: Mon Mar 12 01:15:12.202 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236285>236285</A>
 Terminated at 2007.03.12 01:14:59
<!-- Author: RunControl -->


<!-- Date: Mon Mar 12 01:15:45.526 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236285>236285</A>
TERMINATE: put back silicon as per Ignacio advise
<!-- Author: Jim -->


<!-- Date: Mon Mar 12 01:23:16.946 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
 Activated at 2007.03.12 01:22:52
<!-- Author: RunControl -->


<!-- Date: Mon Mar 12 01:23:17.239 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_4_01 [5,866,678] 
<!-- Author: Jim -->
<!-- Comment: Mon Mar 12 01;35;51 comment by...Jim -->
<pre>during config:

Attention !!!. FERML_HEAP_CORRUPT ERROR !!! 
 
 - Heap Corrupt for crate  b0cot14 
 - Please SHEPHERD crate b0cot14. 

shepherded ok, then it went unresponsive, and shepherded ok again</pre>


<!-- Date: Mon Mar 12 01:49:25.877 2007 -->
<pre>the new TevMon was still in alarm and with the permit revoked, so Ignacio, the Silicon expert, assissted me in granting the permit back.</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->



<!-- Date: Mon Mar 12 02:02:42.096 2007 -->
<pre>The ErrorHandler was printing alot of the following lines since last run and during this run: 
 
(MLW) b0pcal05:Messenger:Mar 12, 2007 01:46:37 AM-&gt;No message sent to r/o [S_objLib_OBJ_UNAVAILABLE] 
 
I have shepherded it and the problems seem gone. While investigating plots and web pages for this, we got the losses and I forgot about it.</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->


<!-- Date: Mon Mar 12 02:03:29.131 2007 -->
<pre>Attention!!!. FERML_TDC_NOT_READY_GLOBAL !!!  
 - Timeout on global TDC Done  for crate b0imu00    
  
 AUTO HRR will be issued.   
  
 If HRR does NOT help,  
 please SHEPHERD the offending TDC crate.   
 
autoHRR ok</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->
<!-- Comment: Mon Mar 12 05;19;06 comment by...Jim -->
<pre>done TO in imu00, autoHRR ok</pre>
<!-- Comment: Mon Mar 12 05;48;09 comment by...Jim -->
<pre>again</pre>
<!-- Comment: Mon Mar 12 07;54;44 comment by...Jim -->
<pre>again</pre>
<!-- Date: Mon Mar 12 02:17:52.119 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
fwd: <tt>West IN/OUT MP Occupancy empty.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->


<!-- Date: Mon Mar 12 02:17:53.428 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
pr: <tt>Low stats to check CCR occupancy.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->


<!-- Date: Mon Mar 12 02:17:54.411 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
general: <tt>CO checklist completed.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->



<!-- Date: Mon Mar 12 02:19:24.271 2007 -->
<pre>the PSH HV bars turned yellow again in HV summary, and the crate7 was flashing red. I searched the elog and this has happened before: 
 
*********** 
Sat Jan 6 01:18:14 
 
PSH kept going yellow in the iFix voltage summary.  
Found that CAEN Crate 8 had a flashing red diagnostic box.    
No alarms went off.  Greg used the cheat sheet upstairs to  
fix the problem.  Willis called and said one channel probably  
went out of tolerance and there is no need to worry. 
 
 - neu 
************ 
 
So it might be the same problem with a channel out of tolerance. I will email Willis just to be sure, and I won't go upstairs to look for the cheat sheet.</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->
<!-- Comment: Mon Mar 12 14;17;47 comment by...Willis -->
CAEN crate 7 slot 3 ch 4 off by 10%. This means the calibration
is off. Maybe it will go away on its own. If not, it will have
to be replaced. Luckily, plug CAEN crate 7 is NE lower.
<!-- Date: Mon Mar 12 02:23:08.003 2007 -->
<pre>Silicon cooling info: 
 
FT_PCE  FT_PCW PT_PCW PT_PCW_1A PCW_2A    
 1.73     2.44   4.97    22.17    15.28</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->
<!-- Comment: Mon Mar 12 06;16;22 comment by...Jim -->
<pre>FT_PCE  FT_PCW PT_PCW PT_PCW_1A PCW_2A    
 1.74     2.38   4.85    22.17    15.87</pre>
<!-- Date: Mon Mar 12 02:43:41.051 2007 -->
<pre>in Daqmon, the CLC+BSC+MNP+RSP button has turned red due to the fact that fcal00 went unresponsive. We don't use that crate right now so nothing special is going on.</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->


<!-- Date: Mon Mar 12 03:02:20.480 2007 -->
<pre>Done TO. for crate/s:  
    b0pcal05    
 
shepherded ok</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->


<!-- Date: Mon Mar 12 03:50:53.513 2007 -->
<pre>ProcMon error: RTserver didn't check ok. 1 potentially stuck port: b0eproxy2. Rechecked and now fine.</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->

<!-- Date: Mon Mar 12 04:03:06.644 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284>236284</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=236284 target=_runSummary>236284</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->



<!-- Date: Mon Mar 12 04:03:28.649 2007 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2007&action=view&page=-2695&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2007&action=view&page=-2695  width='300' align='top'></a></td><td>WHA has a new hot spot (Eta:W7-Phi:9R). 
Emailed to experts.</td></tr></table>
<!-- Author: Kazuhiro -->

<!-- Date: Mon Mar 12 04:13:09.839 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
fwd: <tt>West IN/OUT MP Occupancy empty.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->


<!-- Date: Mon Mar 12 04:13:10.172 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
general: <tt>CO checklist completed.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->



<!-- Date: Mon Mar 12 04:40:23.943 2007 -->
<pre>Heart Beat alarm for CLC/CPR/MNP/BSC. The monitoring program on the clc pc downstairs crashed, restarted it, had to turn on CPR HV as well. Now running fine</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->

<!-- Date: Mon Mar 12 04:45:28.401 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236284>236284</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
MNP crate b0fcal00 is out
</PRE>
<!-- Author: cdfscico -->




<!-- Date: Mon Mar 12 05:39:58.723 2007 -->
<pre>Done TO. for crate/s:  
    b0pcal04    
 
shepherded ok</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->


<!-- Date: Mon Mar 12 05:43:51.291 2007 -->
<pre>Trigger Inhibit is set from the components: 
 VME Power  		[147] VME Crate Power Supplies from iFix 
 
 
something was wrong with 1RR18H_2</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->


<!-- Date: Mon Mar 12 06:06:09.458 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
fwd: <tt>West IN/OUT MP Occupancy empty.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->


<!-- Date: Mon Mar 12 06:06:11.729 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
general: <tt>CO checklisk completed. No new problems.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->




<!-- Date: Mon Mar 12 06:48:27.104 2007 -->
<pre>PSM alarm: 1RR21F_2 had fan status 173 (not ok), it recovered quickly and now we're running</pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->


<!-- Date: Mon Mar 12 07:14:05.899 2007 -->
<pre>for a minute or so the Gamma HV bars in HV summary web page turned red, but the Gamma button in Global alarms remained green. Now everything is green. </pre>
<!-- Author: Jim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->
<!-- Comment: Mon Mar 12 07;25;24 comment by...Jim -->
<pre>now the Gamma button is red and the voice said node unreachable. The list of voltages showed a bunch of @ symbols. Now everything is ok. This was due to some Ifix Pc upgrade.</pre>

<!-- Date: Mon Mar 12 07:30:54.157 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
fwd: <tt>West IN/OUT MP Occupancy empty.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->


<!-- Date: Mon Mar 12 07:30:55.879 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286>236286</A>
general: <tt>CO checklist done.</tt>
<!-- Author: Consumer Operator( Kazuhiro Yamamoto) -->



<!-- Date: Mon Mar 12 07:50:39.047 2007 -->
<font color=red>Shift Summary: </font><pre>Inherited run 236284 with lumi = 99. 
 
Tevatron suffered an RF trip in station 4. 
This caused high abort gap losses, and a TevMon alarm, 
so we set silicon to standby.  The losses calmed down, 
and silicon experts were happy to resume with silcon. 
 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   236284-236286
Delivered Luminosity   1844.50  
Acquired Luminosity    1537.76  
Efficiency             83.4
   
</td></tr></table></pre>
<!-- Author: Rob Snihur -->


<!-- Date: Mon Mar 12 07:51:15.907 2007 -->
TrigMon crashed. Restarted it successfully.
<!-- Author: Kazuhiro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=236286' target=_top>236286</a>) -->