<!-- Title: Owl shift. Fri Oct 26, 2007 -->
<!-- SciCo: David Stuart -->
<!-- DAQAce: Adrian Buzatu -->
<!-- MonAce: &nbsp; -->
<!-- CO: &nbsp; -->
<!-- OpManager: Steve Hahn -->
<!-- Notes: -->
<pre>TeV studies. Rumors of a possible access.</pre>


<!-- Date: Fri Oct 26 00:15:57.318 2007 -->
Fog horn alarm followed by voiced "Cryo alarm." No problem shown on cryo page. Cryo guy did not get an alarm and suspects this is spurious. We'll note it to see if they repeat.
<!-- Author: stuart :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252676' target=_top>252676</a>) -->
<!-- Comment: Fri Oct 26 05;13;49 comment by...DS -->
We got an HVAC audible alarm but iFix shows nothing amiss.
<!-- Date: Fri Oct 26 00:21:14.514 2007 -->
Beginning of shift summary: 
-protons and antiprotons in the Tevatron 
-can run L2Torture, Cosmics and non-QT calibrations 
-should call Silicon if more than 4 hours of quiet time 
-Running Cosmics 
-Monitors run smoothly 
-In ProcMon SVT SPY Monitor still disabled. 
-2 PC farms in L3 appear red 
-HB purple for ICI in Global Alarm turned back to green on its own 
-on Event Display one can see noice in CMX, we will not turn high voltage off and on again (which would be the solution to remediate this, I understand) as there is a risk of trip  
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252676' target=_top>252676</a>) -->


<!-- Date: Fri Oct 26 00:22:46.775 2007 -->
RXPT and Timeout on global TDC Done  for crate b0cot1 Errors on COT18 crate errors. 3 Automatic HRRs and 2 manual HRRs finally solved the problem. I did not have to shepherd it.  
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252676' target=_top>252676</a>) -->


<!-- Date: Fri Oct 26 00:33:17.223 2007 -->
A few minutes ago ICICLE turned red in ProcMon again, but everything is green in Global Alarms and ICICLE program seems to be working on SOLENOID2. Joachim advised on patience. I will see what happens next. 
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252676' target=_top>252676</a>) -->
<!-- Comment: Fri Oct 26 00;37;30 comment by...Adrian -->
Now we got a second voice alarm "ProcMon error" and HB turned purple in ICI in Global Alarms. Also, a voice alarm "Error Heart Beat" came from Global Alarms about the same error.

<!-- Comment: Fri Oct 26 01;02;51 comment by...Adrian -->
When checking ICICLE in ProcMon, it appears as "not updating". The Icicle program window in SOLENOID2 appeared as not updating since 12.05 am (50 minutes ago). Meanwhile there were HB turning purple and no message about this appeared in the window. So I closed the window with Control-C in order to restart Icicle again. But <b> Surprise! </b> At this point the ProcMon Icicle appeared again and no Icicle program was running. HB is still purple. We started the program Icicle again and we are waiting to reproduce the problem. 
<!-- Comment: Fri Oct 26 01;03;32 comment by...Adrian -->
I meant that ICICLE was OK in ProcMon though no Icicle problem was actually running. Strange!
<!-- Comment: Fri Oct 26 01;17;30 comment by...Adrian -->
After a few minutes when Icicle program loaded, the HB on ICI turned red. Icicle program seems to work (always shows new things on the screen) and ICICLE check in Procmon is OK. We will see if the HB error appears again.
<!-- Comment: Fri Oct 26 07;10;06 comment by...Stuart -->
The error has not recurred.
<!-- Comment: Fri Oct 26 09;42;34 comment by...Steve Hahn -->
<p>I looked for anything unusual that might have caused this.
Did not find any network problems, and no virus scan has been
done since 09/22.  So far, no luck.</p>
<!-- Comment: Fri Oct 26 10;03;21 comment by...Steve Hahn -->
But see this 
<a href=
'/cgi/elog/elog.pl?nb=2007&action=view&page=830&frame=2&anchor=10:02:00.437'>comment</a>. 

<!-- Date: Fri Oct 26 00:41:35.006 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252676>252676</A>
 Terminated at 2007.10.26 00:41:03
<!-- Author: RunControl -->


<!-- Date: Fri Oct 26 00:42:57.511 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252676>252676</A>
TERMINATE: Cosmics have run for about 300k events. During the day they could not run L2_Torture, as experts were working on that, but we can now and we will do this.
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 00:46:05.903 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252677>252677</A>
 Activated at 2007.10.26 00:44:01
<!-- Author: RunControl -->


<!-- Date: Fri Oct 26 00:46:06.991 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252677>252677</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,985,706] 
<!-- Author: Adrian Buzatu -->



<!-- Date: Fri Oct 26 00:46:58.522 2007 -->
Actually, I started another run of cosmics by mistake. Some monitors are stuck. I will restart all displays and monitors to see if they run smootly and pick up the new run number.
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252677' target=_top>252677</a>) -->




<!-- Date: Fri Oct 26 01:06:19.087 2007 -->
Restarted successfully all monitors and displays and they picked the new run number(except SVX, of course, as SVX HV is off). TrigMon did not start from "Start all Monitors", I had to start it individually. I will start L2_Torture in order to test this as well.
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252677' target=_top>252677</a>) -->
<!-- Comment: Fri Oct 26 01;07;31 comment by...Adrian -->
CMX noice still on Event Display.
<!-- Date: Fri Oct 26 01:07:04.835 2007 -->
Also, in this new cosmics run, the 2 L3 farms that appeared (disabled) do not appear any more, they are blue and OK. 
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252677' target=_top>252677</a>) -->


<!-- Date: Fri Oct 26 01:09:39.148 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252677>252677</A>
 Terminated at 2007.10.26 01:07:40
<!-- Author: RunControl -->


<!-- Date: Fri Oct 26 01:09:40.236 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252677>252677</A>
TERMINATE: to start L2_Torture.
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 01:13:07.045 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678>252678</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,986,705] 
<!-- Author: Adrian Buzatu -->




<!-- Date: Fri Oct 26 01:22:20.948 2007 -->
TrigMon did not transit smootly to the new L2_Torture run. It kept compaining about "Fatal Error: SIXD_StorableBank   SIXD not found". I believe we had this problem a few days ago and the monitor was modified to igore this error. And the monitor did work with Cosmics. So we try to reproduce the problem. I tried to restart TrigMon. It took a long time to close (maybe 3 minutes). Now I start it again.
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678' target=_top>252678</a>) -->
<!-- Comment: Fri Oct 26 01;39;03 comment by...Adrian -->
For many minutes TrigMon kept giving the same error. We were thinking about starting a new cosmic run to see if the SVX crate error still appears (if it still appears and TrigMon is running, than this error is ignored and would not be the cause of TrigMon not running). But after some minutes, TrigMon did start running also for L2_Torture, picking up the corrent run number.
<!-- Date: Fri Oct 26 01:30:05.873 2007 -->
Three L2decision time outs in the last few minutes. All auto HRR'd successfully.
<!-- Author: stuart :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678' target=_top>252678</a>) -->
<!-- Comment: Fri Oct 26 01;52;43 comment by...Adrian -->
A few more identical errors, Automatic HRRs solved them,


<!-- Date: Fri Oct 26 01:53:49.489 2007 -->
I wrote this email to TrigMon expert (mailto: nageshk@fnal.gov). I see that TrigMon is   
slow to die when closed, but it finally does it, does not start on it's    
own from "Start all Monitors" but starts when I start it individually, it 
takes about 5-10 minutes to finally appear as running in the display 
windows, but then it does catch correctly the run number both in Cosmics   
and L2_Torture. In all cases it gives this error "SIXD not found" and I    
guess this is ignored as SVX HV is off currently.  
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678' target=_top>252678</a>) -->
<!-- Comment: Fri Oct 26 02;32;11 comment by...stuart -->
Actually, the reason is not related to the state of the HV but rather that the SVX crates are not included in the run, so there is no SVX data. 
<!-- Date: Fri Oct 26 02:06:23.129 2007 -->
<pre>Monday day shift  Greg Feild asked us to use partition 3 for about a week. Tuesday day shift ACE switched back to partition 0 after some tests on partition 0,1,2,3 by RXPT people. Since no explanation by Greg or ACE about this, I guess that maybe ACE forgot to continue using partion 3 and since then we have been using only partition 0. I just wrote an email to Greg asking him about this.  
 
&lt;a href= 
'/cgi/elog/elog.pl?nb=2007&action=view&page=818&frame=2&anchor=09:25:23.173'&gt; 
Here &lt;/a&gt; is Greg Feild initial request  
 
&lt;a href= 
'/cgi/elog/elog.pl?nb=2007&action=view&page=818&frame=2&anchor=09:25:23.173'&gt;Here are the XTPR studies&lt;/a&gt;. You can click the previous run numbers and see in the details page that they were using partition 3 and the following runs partition 0. </pre>
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678' target=_top>252678</a>) -->
<!-- Comment: Fri Oct 26 02;08;19 comment by...Adrian -->
<b>Please read the next entry directly, the links work there.</b>
<!-- Date: Fri Oct 26 02:07:34.510 2007 -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>Monday day shift  Greg Feild asked us to use partition 3 for about a week. Tuesday day shift ACE switched back to partition 0 after some tests on partition 0,1,2,3 by RXPT people. Since no explanation by Greg or ACE about this, I guess that maybe ACE forgot to continue using partion 3 and since then we have been using only partition 0. I just wrote an email to Greg asking him about this.  
 
&lt;a href= 
'/cgi/elog/elog.pl?nb=2007&action=view&page=818&frame=2&anchor=09:25:23.173'&gt; 
Here &lt;/a&gt; is Greg Feild initial request  
 
&lt;a href= 
'/cgi/elog/elog.pl?nb=2007&action=view&page=818&frame=2&anchor=09:25:23.173'&gt;Here are the XTPR studies&lt;/a&gt;. You can click the previous run numbers and see in the details page that they were using partition 3 and the following runs partition 0. </pre></td></tr></td></table>
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678' target=_top>252678</a>) -->
<!-- Comment: Fri Oct 26 02;12;20 comment by...Adrian -->
Ups, they do not work here either. 
<!-- Date: Fri Oct 26 02:11:43.116 2007 -->
Running L2_Torture smoothly. Current Total Deadtime: 11.22% and average 9.86%. 
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678' target=_top>252678</a>) -->




<!-- Date: Fri Oct 26 03:28:31.221 2007 -->
About three L2 Decision T0s since the last elog entry, all recovered by automatic HRRs. Also, 9.81% L2 current deadtime and 10.19% average.
<!-- Author: Adrian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678' target=_top>252678</a>) -->

<!-- Date: Fri Oct 26 04:39:26.863 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252678>252678</A>
TERMINATE: L2 trigger rate drops to zero. A few HRRs could not solve the problem, it would remain in red Active state. We will do non quiet time calibrations now.
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 05:13:03.574 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252679>252679</A>
ACTIVATE: RunSet: ACE_TOF_QIE_NEW 
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 05:13:04.463 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252679>252679</A>
TERMINATE: done
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 05:13:05.131 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252679>252679</A>
CALIBRATION: <U><B>tofqie</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#tofqie target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Fri Oct 26 06;43;25 comment by...A -->
Good.
<!-- Date: Fri Oct 26 05:13:06.919 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252680>252680</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,986,705] 
<!-- Author: Adrian Buzatu -->



<!-- Date: Fri Oct 26 06:04:04.089 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252680>252680</A>
TERMINATE: to do calibrations once more.
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:09:39.913 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252681>252681</A>
ACTIVATE: RunSet: SMQIECALIB 
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:09:40.354 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252681>252681</A>
TERMINATE: done
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:09:41.488 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252682>252682</A>
ACTIVATE: RunSet: ACE_LED_ALL 
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:11:49.572 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252682>252682</A>
CALIBRATION: <U><B>cemled</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#cemled target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Fri Oct 26 06;44;05 comment by...Adrian -->
Good.
<!-- Date: Fri Oct 26 06:11:50.236 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252682>252682</A>
TERMINATE: done
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:11:52.329 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252681>252681</A>
CALIBRATION: <U><B>smx</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#smx target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Fri Oct 26 06;44;39 comment by...Adrian -->
Good.
<!-- Date: Fri Oct 26 06:11:53.140 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252683>252683</A>
ACTIVATE: RunSet: COTCALIB_ACE 
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:16:34.614 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252683>252683</A>
CALIBRATION: <U><B>cot</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#cot target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Fri Oct 26 06;45;05 comment by...Adrian -->
BAD
<!-- Date: Fri Oct 26 06:16:35.414 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252684>252684</A>
ACTIVATE: RunSet: XEF_ALL 
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:16:36.250 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252684>252684</A>
CALIBRATION: <U><B>calxef</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#calxef target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Fri Oct 26 06;45;26 comment by...Adrian -->
Good.
<!-- Date: Fri Oct 26 06:16:37.698 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252684>252684</A>
TERMINATE: done
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:20:48.138 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252685>252685</A>
ACTIVATE: RunSet: CALIB_MUON_ALL1 
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:20:49.822 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252685>252685</A>
TERMINATE: done
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:28:37.180 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252686>252686</A>
CALIBRATION: <U><B>toftac</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#toftac target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Fri Oct 26 06;45;56 comment by...Adrian -->
BAD
<!-- Date: Fri Oct 26 06:28:38.289 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252686>252686</A>
TERMINATE: done
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:28:39.977 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252686>252686</A>
ACTIVATE: RunSet: TOF_TAC_CALIB 
<!-- Author: Adrian Buzatu -->


<!-- Date: Fri Oct 26 06:42:34.686 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252687>252687</A>
 Activated at 2007.10.26 06:37:50
<!-- Author: RunControl -->


<!-- Date: Fri Oct 26 06:42:35.063 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=252687>252687</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,985,706] 
<!-- Author: Adrian Buzatu -->









<!-- Date: Fri Oct 26 07:55:06.207 2007 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>RunNumber</TH>
<TH>DataType</TH>
<TH>PhysicsTable</TH>
<TH>Begin</TH>
<TH>End</TH>
<TH>LiveTime</TH>
<TH>L3Accepts</TH>
<TH>DelivLumi nb<SUP>-1</SUP></TH>
<TH>LiveLumi nb<SUP>-1</SUP></TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>
<TR><TD>Totals</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD>
<TD>07:55:00</TD>
<TD ALIGN=CENTER>::</TD>
<TD ALIGN=RIGHT></TD>               <TD ALIGN=RIGHT></TD>
<TD ALIGN=RIGHT></TD>
<TD> </TD>
<TD> </TD>
<TD> </TD></TR>

</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Fri Oct 26 08:01:04.375 2007 -->
<font color=red>Shift Summary: </font><pre>Just twiddled our thumbs. 
The only issues are a heart beat problem that was apparently fixed by restarting the program, and a question of what the correct partition is. 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   none
Delivered Luminosity   0  
Acquired Luminosity    0  
Efficiency             0
   
</td></tr></table></pre>
<!-- Author: David Stuart -->