<!-- Title: Day shift. Sun Dec  9, 2007 -->
<!-- SciCo: Ron Moore -->
<!-- DAQAce: Pedro Ttito-Guzman -->
<!-- MonAce: &nbsp; -->
<!-- CO: Maurizio Iori -->
<!-- OpManager: Max Goncharov -->
<!-- Notes: -->
<pre>HEP store 5788 spinning, CDF lumi = 51 ub^-1/s.  Taking data without pcal00.  150 E10 pbars in the Recycler</pre>


<!-- Date: Sun Dec 9 08:12:16.148 2007 -->
Done TO for crate b0clc00. Auto-HRR fixed it.
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254954' target=_top>254954</a>) -->
<!-- Comment: Sun Dec  9 08;17;08 comment by...Pedro -->
Again
<!-- Date: Sun Dec 9 08:14:13.243 2007 -->
Just to repeat from last shift's notes, Steve Hahn is coming to help diagnose pcal00 and west plug PSM problem.
<!-- Author: ronmoore -->


<!-- Date: Sun Dec 9 08:15:59.814 2007 -->
L2 Decision TO. Auto-HRR recovered.
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254954' target=_top>254954</a>) -->
<!-- Comment: Sun Dec  9 08;42;18 comment by...Pedro -->
Again
<!-- Comment: Sun Dec  9 09;41;20 comment by...Pedro -->
Again
<!-- Date: Sun Dec 9 08:29:05.807 2007 -->
<b>MCR Plan</b><br> 
The AD RunCo will set the store termination time later this morning.  D0 has already asked for an access after this store, and I told MCR we may need an access as well pending additional investigation of pcal00.  In addition, AD Cryo would like to check on a Tevatron wet engine between stores.
<!-- Author: ronmoore -->



<!-- Date: Sun Dec 9 08:52:58.592 2007 -->
Data taking has stopped.  Ace asked me to page L3/EVB because it looks like the EVB did not make it through a HALT.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 08;53;51 comment by...ronmoore -->
Khaldoun has responded.  Pedro is talking to him now.

<!-- Comment: Sun Dec  9 08;54;44 comment by...ronmoore -->
The EVB Ace Panel is not responding.  Khaldoun is coming in to assist.


<!-- Comment: Sun Dec  9 09;17;55 comment by...ronmoore -->
We are taking data again.
<!-- Comment: Sun Dec  9 09;20;58 comment by...khaldoun -->
<pre>It looks like at least 2 things happened simultaneously or near the same time.  That RC got stuck in going to the HALT state means that some client was unresponsive, and in this case, I expect that one of the SCPUs had crashed.

However, the Ace Control Panel, which would normally be used to diagnose this kind of problem, had also crashed on b0eproxy01.  When I logged into eproxy01, the process was no longer running (some java windows remained on b0control20, an artifact).  

Action taken:

--Get RC to START state, using the manual override for every transition (since no transition could complete successfully until we could clean up EVB).
--Restart the ACP
--Clean up EVB

This was the fix.</pre>
<!-- Date: Sun Dec 9 08:58:27.072 2007 -->
<pre>We had this chain of errors:  
 
 L3 trigger rate has dropped to 0 Hz!!!  
     Events are not being logged !!! 
 
 L2 trigger rate has dropped to 0 Hz!!!  
     Events are not being logged !!!  
 
 Done TO. for crate/s:  
    b0cot14   
 
Error Detected in ProcMon. 
 
</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254954' target=_top>254954</a>) -->

<!-- Date: Sun Dec 9 09:08:55.527 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254954>254954</A>
 Terminated at 2007.12.09 09:08:02
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 09:08:56.510 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254954>254954</A>
TERMINATE: end run
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 09:17:54.248 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254955>254955</A>
 Activated at 2007.12.09 09:16:07
<!-- Author: RunControl -->




<!-- Date: Sun Dec 9 09:32:07.252 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254955>254955</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_5_01 [2,1010,710] 
<!-- Author: Pedro -->
<!-- Comment: Sun Dec  9 09;35;01 comment by...ronmoore -->
Actually, we lost 5 min because PES LV was inhibiting the run.



<!-- Date: Sun Dec 9 09:42:57.812 2007 -->
Done TO for crate b0cot14. Manual HRR fixed it.
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254955' target=_top>254955</a>) -->


<!-- Date: Sun Dec 9 09:48:11.723 2007 -->
Paged Larry (unintentionally) regarding the pcal00 problem.  He suggested paging Mark Mattson to discuss.  Steve really is on his way in now, too.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 09;56;34 comment by...ronmoore -->
Mark will investigate and call us back.


<!-- Comment: Sun Dec  9 10;04;48 comment by...m mattson (ADMEM) -->
<p>
Responded to the ADMEM pager.  I looked at the error log for the last run with pcal00 included (254953), and there was something funny about slot 21.  When I logged into the crate, the CAFE ID numbers for slot 21 were messed up, which is a symptom of ADMEM configuration file corruption.  I reprogrammed slot 21 as PHA from the QIETEST program.
<p>
Unless there are other problems, the crate should be okay for data taking.
<p>
At the beginning of new runs, the error log will have a message about pcal00 slot 21 not being properly configured.  All that means is that the slot was properly configured by something other than RunControl; ignore the message.
<!-- Date: Sun Dec 9 09:57:24.523 2007 -->
Pedro asked me to page L3 again.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 10;19;55 comment by...ronmoore -->
It looks like the same problem as earlier this shift.
<!-- Date: Sun Dec 9 10:04:38.103 2007 -->
<b>MCR Plan</b><br> 
Terminate current store at 11:00, start D0 2-3 hour access. 
I mentioned that we don't know if we need an access yet, so they 
may delay store termination if we need more time to diagnose pcal00.
<!-- Author: ronmoore -->



<!-- Date: Sun Dec 9 10:05:29.253 2007 -->
In the meantime, Mark Mattson said we can try putting pcal00 back into the run.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 10;09;19 comment by...m mattson (ADMEM) -->
All runs with pcal00 excluded will be marked bad anyways (missing 4 phi wedges for PEM and PHA calorimeter).

<!-- Date: Sun Dec 9 10:11:00.922 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254955>254955</A>
 Terminated at 2007.12.09 10:10:18
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 10:11:01.302 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254955>254955</A>
TERMINATE: end run
<!-- Author: Pedro -->



<!-- Date: Sun Dec 9 10:18:30.812 2007 -->
Ace still on phone with Khaldoun.  Steve here looking at PSM.  Gas alarm - cryo taking care of a low flow.  Silicon PS#2 auto-hockerization - waiting to recover until Ace is free and RC is in known state.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 10;27;13 comment by...Ricardo -->
Thanks for clarifying, I was about to call you guys.

<!-- Date: Sun Dec 9 10:32:58.422 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254956>254956</A>
TERMINATE: end run
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 10:39:25.180 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254957>254957</A>
 Activated at 2007.12.09 10:39:21
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 10:39:49.101 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254957>254957</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_5_01 [2,1010,710] 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 10:50:37.234 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254957>254957</A>
 Terminated at 2007.12.09 10:50:31
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 10:50:52.706 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254957>254957</A>
TERMINATE: end run
<!-- Author: Pedro -->
<!-- Comment: Sun Dec  9 10;51;52 comment by...khaldoun -->
<pre>Why was this run ended?</pre>


<!-- Comment: Sun Dec  9 10;59;31 comment by...ronmoore -->
Forgot to include pcal00.
<!-- Comment: Sun Dec  9 11;00;20 comment by...khaldoun -->
<pre>Thanks</pre>
<!-- Date: Sun Dec 9 10:55:04.522 2007 -->
OK, here's a quick summary of the latest flurry of activity.  Khaldoun helped recover the EVB.  Once RC was back in start, the Ace recovered silicon from the PS #2 auto-hocker;  I unmarked the 28 pinkies caused by turning on that PS.  (Ricardo told me later about the "Unmark All".  I had unmarked them individually.)  We started taking data again, but had forgotten to reinclude pcal00.  (D'oh!)  We are now including pcal00.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 10;56;03 comment by...ronmoore -->
I have asked MCR to delay store termination until we can run with pcal00 for a while to see if it is OK.  We don't yet know if we need an access.
<!-- Comment: Sun Dec  9 10;59;00 comment by...ronmoore -->
There is still a PES LV inhibit.  Steve was unable to recover the PSM for the west plug, so we don't know if there is a bad shower-max crate.  After collecting some data, Steve will look at YMON plots to check for a problem.
<!-- Comment: Sun Dec  9 10;59;54 comment by...khaldoun -->
<pre>I want to make it explicit that the EVB problem was a very quick fix.  We ended the run, and cleaned up EVB.  Downtime associated with that should not go above 15min for any reason.  

After that, there was a lot of time waiting for the FIB problem, and generally dealing with coordination in the control room.  It sounds like (from what I could tell on the phone) that the shift crew was assaulted by 2-3 problems at once :-).  

Let's hope the rest of the shift if more restful for you.</pre>
<!-- Date: Sun Dec 9 10:55:15.171 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254958>254958</A>
 Activated at 2007.12.09 10:55:04
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 10:55:16.303 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254958>254958</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_5_01 [2,1010,710] 
<!-- Author: Pedro -->








<!-- Date: Sun Dec 9 11:04:49.387 2007 -->
PPR is missing in  W7,8, 21,22.  This is pcal01 and pcal05, you might try recalibrating. 
If that fails power cycle  ....
<!-- Author: Larry N :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254958' target=_top>254958</a>) -->


<!-- Date: Sun Dec 9 11:06:25.890 2007 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2007&action=view&page=-7269&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2007&action=view&page=-7270  width='300' align='top'></a></td><td>ppr for current run</td></tr></table>
<!-- Author: Larry N -->

<!-- Date: Sun Dec 9 11:12:39.922 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254958>254958</A>
 Terminated at 2007.12.09 11:12:14
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 11:12:40.791 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254958>254958</A>
TERMINATE: end run
<!-- Author: Pedro -->



<!-- Date: Sun Dec 9 11:16:06.508 2007 -->
Steve and Larry chatted about the plug.  Steve will lead a calibration and power-cycle to fix the (new?) PPR problem.  Graciously, MCR has allowed us even more time to diagnose before terminating store 5788.
<!-- Author: ronmoore -->

<!-- Date: Sun Dec 9 11:17:47.597 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254959>254959</A>
ACTIVATE: RunSet: SMQIECALIB 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 11:23:00.619 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254959>254959</A>
CALIBRATION: <U><B>smx</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#smx target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 11:25:37.262 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254959>254959</A>
TERMINATE: Try again, this time with FRAM enabled

<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 11:29:36.585 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254960>254960</A>
CALIBRATION: <U><B>smx</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#smx target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 11:40:50.407 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254961>254961</A>
ACTIVATE: RunSet: SMQIECALIB 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 11:43:47.723 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254961>254961</A>
CALIBRATION: <U><B>smx</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#smx target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Sun Dec  9 11;44;38 comment by...Larry N -->
they all look good to me  


<!-- Comment: Sun Dec  9 11;57;07 comment by...Steve Hahn -->
<p>Yies, I agree, except for one detail.  Looking at DBANA,
the (slopt 7)/(slope 6) plot always comes up with a red border.
However, to my eye, the plot looks fine.
  But we're having trouble downloading the 
calibration.  First, Larry forgot to tell me I need to set the
expert option for downloading the FRAMs.  Second, twice the
CALIB_LOAD step failed on waiting for b0pcal08.  However, we 
could ping b0pcal08 and on sheparding and rebooting this crate,
it looked from the console as if after rebooting, the QIE download
finished.  Weird.  So now we are trying physics again.</p>
<!-- Date: Sun Dec 9 11:55:19.545 2007 -->
During the plug calibrations, there was a silicon pinky DVDD SVX B0W3L3.  Ricardo said not to worry about it, just do a cold-start when calibrations are wrapped up.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 12;11;43 comment by...ronmoore -->
This current returned to the expected range after the cold-start.
<!-- Date: Sun Dec 9 11:56:42.101 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254962>254962</A>
 Activated at 2007.12.09 11:56:38
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 11:57:01.936 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254962>254962</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_5_01 [2,1010,710] 
<!-- Author: Pedro -->
<!-- Comment: Sun Dec  9 12;00;44 comment by...Pedro -->
We are taking data again.
<!-- Date: Sun Dec 9 11:59:23.648 2007 -->
Done T0 for crate b0svt00. Auto-HRR fixed it.
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254962' target=_top>254962</a>) -->


<!-- Date: Sun Dec 9 12:07:58.595 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254962>254962</A>
 Terminated at 2007.12.09 12:07:37
<!-- Author: RunControl -->



<!-- Date: Sun Dec 9 12:08:00.848 2007 -->
woo! plug shower max looks ok
<!-- Author: Larry N :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254962' target=_top>254962</a>) -->

<!-- Date: Sun Dec 9 12:08:14.384 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254954>254954</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=254954 target=_runSummary>254954</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->



<!-- Date: Sun Dec 9 12:10:08.892 2007 -->
Everything looks OK in the west pcal now, except for the power supply monitoring.  We are preparing for end-of-store now.  Steve will keep looking at the PSM.  It is possible we would need an access to fix the monitoring, but we will decide tha later.  Thanks to D0 and MCR for their patience.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 12;17;54 comment by...ronmoore -->
Steve is calling Roberto for assistance with the PSM problem.
<!-- Comment: Sun Dec  9 12;37;04 comment by...ronmoore -->
Joy!  PSM is working again!  (Steve will tell the tale.)
<!-- Comment: Sun Dec  9 12;53;25 comment by...ronmoore -->
No need to access at this time.
<!-- Date: Sun Dec 9 12:15:00.244 2007 -->
Informed MCR that we are ready for store termination.
<!-- Author: ronmoore -->
<!-- Comment: Sun Dec  9 12;16;45 comment by...ronmoore -->
The store is dead. Long live the store.



<!-- Date: Sun Dec 9 12:19:53.598 2007 -->
<p>We can't get a hold of Dervin, and I hate to get Roberto in  
here again since he was here yesterday, so I was trying to figure 
out what Roberto did yesterday to make the west plug come back. 
Tried looking at the program, found no problems earlier.  Then 
just now tried just restarting program (volt.exe)--no good.  Then 
tried rebooting VOLTMAN--still no good.  Calling Roberto.</p>
<!-- Author: Steve Hahn -->


<!-- Date: Sun Dec 9 12:20:26.795 2007 -->
We expect to have > 2 hrs of quiet time.  D0 is going on access (2-3 hrs) and AD Cryo is working on the C2 wet engine (up to 4 hrs).
<!-- Author: ronmoore -->


<!-- Date: Sun Dec 9 12:42:06.085 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254964>254964</A>
 Activated at 2007.12.09 12:41:18
<!-- Author: RunControl -->



<!-- Date: Sun Dec 9 12:44:59.837 2007 -->
<table border=0><td><tr><td bgcolor=#e0e030><p>Dervination or Davilanation?</p> 
 
<p>I talked to Roberto, and followed his prescription: </p> 
 
<ol> 
<li>Exit the volt.exe program on VOLTMAN.</li> 
<li>Turn off the entire PSM power supply in rack 1RR06F using 
the front-panel toggle switch.</li> 
<li>Wait one minute.</li> 
<li>Turn the PSM power supply on.</li> 
<li>Restart the volt.exe program by double-clioking on the  
"Volt.exe - .NET" shortcut icon on the desktop on VOLTMAN, 
and clicking the "RUN" button on the display when the program 
starts.</li> 
</ol> 
 
<p>I would recommend trying this procedure if PSM problems  
occur again.  I will be changing the documentation to reflect 
this.</p></td></tr></td></table>
<!-- Author: Steve Hahn :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254963' target=_top>254963</a>) -->
<!-- Comment: Sun Dec  9 12;52;08 comment by...Steve Hahn -->
Oops, did not note that this brought back the west plug SMX crates
which had been bad at least since evening yesterday.  Actually,
Roberto and Dervin have done this for a long time; why it works
when any other order does not is a mystery I would love to solve.
<!-- Date: Sun Dec 9 12:48:58.712 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254964>254964</A>
 Terminated at 2007.12.09 12:48:06
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 12:48:59.097 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254964>254964</A>
TERMINATE: starting quiet time calibrations
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 12:50:34.727 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254965>254965</A>
ACTIVATE: RunSet: ACE_QIE 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 12:52:08.244 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254963>254963</A>
CALIBRATION: <U><B>calqie</B></U> calibration FAILED, please investigate
<!-- Author: CDFCALIB -->



<!-- Date: Sun Dec 9 12:52:44.804 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254965>254965</A>
TERMINATE: Done
<!-- Author: Pedro -->



<!-- Date: Sun Dec 9 12:59:17.411 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254966>254966</A>
ACTIVATE: RunSet: CALIB_CLC_QIE 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 12:59:19.583 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254966>254966</A>
CALIBRATION: <U><B>clcqie</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#clcqie target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 12:59:42.652 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254966>254966</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:08:07.726 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254967>254967</A>
ACTIVATE: RunSet: CALIB_QIE_FWD 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:09:10.084 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254967>254967</A>
CALIBRATION: <U><B>bscqie</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#bscqie target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 13:13:25.171 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254968>254968</A>
ACTIVATE: RunSet: BOTHPLUGS_NORMAL 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:23:31.499 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254968>254968</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:24:55.084 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254967>254967</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:25:41.792 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254955>254955</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=254955 target=_runSummary>254955</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 13:28:54.768 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254969>254969</A>
CALIBRATION: <U><B>pluglaser</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#pluglaser target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 13:28:55.346 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254969>254969</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:28:56.186 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254969>254969</A>
ACTIVATE: RunSet: BOTHPLUGS_MM_C 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:31:23.789 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254970>254970</A>
ACTIVATE: RunSet: BOTHPLUGS_MM 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:33:06.033 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254954>254954</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
PCAL - pcal00 excluded from run
PPR  - pcal00 excluded from run
TOF  - number of bad channels for "TOF East Ave Time" plot is 9
</PRE>
<!-- Author: cdfscico -->


<!-- Date: Sun Dec 9 13:33:59.576 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254970>254970</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:56:54.888 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254971>254971</A>
 Activated at 2007.12.09 13:56:42
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 13:56:55.067 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254971>254971</A>
ACTIVATE: RunSet: ACE_CALORIMETER_MINIMAL  Trigger: SUMET5 [7,983,706] 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 13:59:16.690 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254955>254955</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
PCAL - pcal00 excluded
PPR  - pcal00 excluded
</PRE>
<!-- Author: cdfscico -->


<!-- Date: Sun Dec 9 14:01:22.790 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254971>254971</A>
 Terminated at 2007.12.09 14:01:02
<!-- Author: RunControl -->



<!-- Date: Sun Dec 9 14:03:36.839 2007 -->
Fotis has stopped in to request a CHA calibration.
<!-- Author: ronmoore -->

<!-- Date: Sun Dec 9 14:04:58.754 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254958>254958</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
PPR - blank strips - needs calibration
</PRE>
<!-- Author: cdfscico -->


<!-- Date: Sun Dec 9 14:07:04.138 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254972>254972</A>
ACTIVATE: RunSet: CENTRAL_HADRON_LASER  Trigger: SUMET5 [7,983,707] standard calibration
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:11:29.817 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254972>254972</A>
TERMINATE: finish central hadron calibration - (Fotis)
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:15:53.938 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254973>254973</A>
 Activated at 2007.12.09 14:14:07
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 14:15:54.071 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254973>254973</A>
ACTIVATE: RunSet: ACE_CALORIMETER_MINIMAL  Trigger: SUMET5 [7,983,706] 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:26:18.016 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254973>254973</A>
 Terminated at 2007.12.09 14:26:15
<!-- Author: RunControl -->


<!-- Date: Sun Dec 9 14:27:33.470 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254973>254973</A>
TERMINATE: finish PMT Spike run
<!-- Author: Pedro -->
<!-- Comment: Sun Dec  9 14;28;56 comment by...Pedro -->
L1=33.9 Hz, 30132 events(L2)


<!-- Date: Sun Dec 9 14:30:23.197 2007 -->
Starting Non-quite time calibrations
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254973' target=_top>254973</a>) -->

<!-- Date: Sun Dec 9 14:32:31.051 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254974>254974</A>
ACTIVATE: RunSet: ACE_TOF_QIE_NEW 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:34:10.789 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254974>254974</A>
CALIBRATION: <U><B>tofqie</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#tofqie target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 14:34:11.978 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254974>254974</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:38:23.544 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254975>254975</A>
ACTIVATE: RunSet: SMQIECALIB 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:38:24.888 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254975>254975</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:40:34.145 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254975>254975</A>
CALIBRATION: <U><B>smx</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#smx target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 14:40:35.436 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254976>254976</A>
ACTIVATE: RunSet: ACE_LED_ALL 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:42:27.242 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254976>254976</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:42:28.040 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254976>254976</A>
CALIBRATION: <U><B>cemled</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#cemled target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 14:45:47.162 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254977>254977</A>
ACTIVATE: RunSet: COTCALIB_ACE 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:45:48.405 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254977>254977</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:45:49.393 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254977>254977</A>
CALIBRATION: <U><B>cot</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#cot target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 14:51:10.510 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254978>254978</A>
ACTIVATE: RunSet: XEF_ALL 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:52:07.925 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254978>254978</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 14:52:08.449 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254978>254978</A>
CALIBRATION: <U><B>calxef</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#calxef target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 14:59:31.949 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254979>254979</A>
ACTIVATE: RunSet: CALIB_MUON_ALL1 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 15:01:27.368 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254979>254979</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 15:08:28.714 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254980>254980</A>
ACTIVATE: RunSet: TOF_TAC_CALIB 
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 15:10:20.593 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254980>254980</A>
TERMINATE: Done
<!-- Author: Pedro -->


<!-- Date: Sun Dec 9 15:10:21.718 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254980>254980</A>
CALIBRATION: <U><B>toftac</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#toftac target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Dec 9 15:21:31.019 2007 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254981>254981</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,986,705] 
<!-- Author: Pedro -->



<!-- Date: Sun Dec 9 15:31:33.338 2007 -->
The Tevatron C2 wet engine requires a complete overhaul, so it may be several more hours before the next HEP shot-setup.
<!-- Author: ronmoore -->


<!-- Date: Sun Dec 9 15:52:33.179 2007 -->
<TABLE bgcolor=#cdc9a5> 
<tr><th colspan=2>Totals</th></tr> 
 
<tr><td>Date:</td><td>2007.12.09</td> 
<tr><td>Shift:</td><td>day</td> 
<tr><td>Delivered luminosity:</td><td> 677.6 nb<sup>-1</sup></td></tr> 
<tr><td>Acquired luminosity:</td><td> 237.9 nb<sup>-1</sup></td></tr> 
<tr><td>Efficiency:</td><td> 35.1</td></tr> 
 
</TABLE>
<!-- Author: ronmoore -->


<!-- Date: Sun Dec 9 15:54:41.098 2007 -->
<font color=red>Shift Summary: </font><pre>* Start shift with store 5788, CDF lumi = 51 ub^-1/s.  Taking data without 
pcal00 - data to be marked bad. 
 
* 08:52 - Problem with EVB, data-taking stopped.  Khaldoun investigated, found 
two problems.  Resolved with ace. 
 
* 09:55 - Repeat of same problem with EVB.  Khaldoun guided Ace through the 
fix. 
 
* 10:00 - Mark Mattson reprogrammed pcal00 slot 21 ADMEM; asked us to put 
pcal00 back into the run.  (Too bad he wasn't called earlier.) 
 
* Silicon PS #2 auto-hocker during (2nd) EVB fix.  Recovered silicon PS after 
EVB problem resolved.  The resulting 28 pinkies were unmarked successfully. 
 
* Steve Hahn came in to help with plug problems.  After consultations with 
Karen and Larry, showermax calibrations and putting in pcal00, all was well.   
Steve even fixed plug PSM monitoring with a different procedure (documented in eLog). 
 
* 12:16 - Store 5788 terminated.  Thanks to D0 and MCR for waiting for us 
to investigate the west plug problems. 
 
* Ace completed calibrations. 
 
* Shift ending with Tevatron off for C2 wet engine repair. 
 
</pre> 
<!-- Author: ronmoore -->

<!-- Date: Sun Dec 9 15:55:46.343 2007 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>RunNumber</TH>
<TH>DataType</TH>
<TH>PhysicsTable</TH>
<TH>Begin</TH>
<TH>End</TH>
<TH>LiveTime</TH>
<TH>L3Accepts</TH>
<TH>DelivLumi nb<SUP>-1</SUP></TH>
<TH>LiveLumi nb<SUP>-1</SUP></TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=254954 target=_runSummary>254954</A> x3E3EA</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [2,1010,710]</FONT></TD>
<TD ALIGN=CENTER>05:47:12</TD>
<TD ALIGN=CENTER>09:08:02</TD>
<TD ALIGN=CENTER>02:42:17</TD>
<TD ALIGN=RIGHT>     815,492</TD>   <TD ALIGN=RIGHT>    646.91</TD>
<TD ALIGN=RIGHT>    526.19</TD>
<TD ALIGN=CENTER>0</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=254955 target=_runSummary>254955</A> x3E3EB</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [2,1010,710]</FONT></TD>
<TD ALIGN=CENTER>09:16:07</TD>
<TD ALIGN=CENTER>10:10:18</TD>
<TD ALIGN=CENTER>00:21:09</TD>
<TD ALIGN=RIGHT>      88,789</TD>   <TD ALIGN=RIGHT>    152.96</TD>
<TD ALIGN=RIGHT>     59.89</TD>
<TD ALIGN=CENTER>0</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=254956 target=_runSummary>254956</A> x3E3EC</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [2,1010,710]</FONT></TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>10:32:30</TD>
<TD ALIGN=CENTER>00:00:00</TD>
<TD ALIGN=RIGHT></TD>               <TD ALIGN=RIGHT></TD>
<TD ALIGN=RIGHT></TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>0</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=254958 target=_runSummary>254958</A> x3E3EE</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [2,1010,710]</FONT></TD>
<TD ALIGN=CENTER>10:55:04</TD>
<TD ALIGN=CENTER>11:12:14</TD>
<TD ALIGN=CENTER>00:15:28</TD>
<TD ALIGN=RIGHT>      49,369</TD>   <TD ALIGN=RIGHT>     45.85</TD>
<TD ALIGN=RIGHT>     40.18</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=254962 target=_runSummary>254962</A> x3E3F2</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [2,1010,710]</FONT></TD>
<TD ALIGN=CENTER>11:56:38</TD>
<TD ALIGN=CENTER>12:07:37</TD>
<TD ALIGN=CENTER>00:08:31</TD>
<TD ALIGN=RIGHT>      19,881</TD>   <TD ALIGN=RIGHT>     25.88</TD>
<TD ALIGN=RIGHT>     21.33</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD>Totals</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD>
<TD>15:55:01</TD>
<TD ALIGN=CENTER>03:27:27</TD>
<TD ALIGN=RIGHT>       973,531</TD> <TD ALIGN=RIGHT>    871.60</TD>
<TD ALIGN=RIGHT>    647.60</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD></TR>

</TABLE>
<!-- Author: End of Shift Report -->