<!-- Title: Eve shift. Thu Mar  6, 2008 -->
<!-- SciCo: Joe Boudreau -->
<!-- DAQAce: Hyunsu Lee -->
<!-- MonAce: Hyunsu Lee -->
<!-- CO: Sarah Budd -->
<!-- OpManager: Massimo Casarsa -->
<!-- Notes: -->
<pre></pre>


<!-- Date: Thu Mar 6 16:29:48.309 2008 -->
Plug HV 08 trip. It was red in the check passed. I reset it and restart HV monitoring program. Everything is ok.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->
<!-- Comment: Thu Mar  6 16;42;52 comment by...Hyunsu -->
It happened two more. We paged to willis.
<!-- Comment: Thu Mar  6 17;55;36 comment by...Hyunsu -->
Sorry, check passed for 08 was off when I saw it.
<!-- Date: Thu Mar 6 16:31:09.980 2008 -->
<pre>Muon Triggers were extremely hot at the start of the store. 
This is actually a well-known problem that occurs whenever  
we have a clock glitch (as happened last night).  Many TDCs 
had to be power-cycled (CMU, CMX, BMU).  This problem should  
in principal be identifiable running cosmics prior to the  
store.  At some level, we're a victim of furloughs as Phil  
Schlabach who is away this week usually takes it upon himself  
to take a look at the TDCs after power-cycles/clock glitches. 
We probably should not be relying solely on Phil to handle  
this, though, so we need to make sure that these checks are  
re-integrated into standard checkout procedures. 
 
Note that as part of this round of problems, the tdc in  
slot07 of cmu00 did develop some real problems.  Acutally,  
we saw problems in this board a couple of weeks ago and  
thought that we had eliminated them by swaping the mezzanine  
card.  This time I actually replaced the TDC board in this  
slot with the one from slot14 and put a new spare board into 
slot14.  Things look ok starting with run 259077.  Silicon 
will be re-integrated shortly.  </pre>
<!-- Author: Eric James -->


<!-- Date: Thu Mar 6 16:33:05.033 2008 -->
DQMon error for IMU.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->
<!-- Comment: Thu Mar  6 16;53;58 comment by...Hyunsu -->
PHA alarm
<!-- Comment: Thu Mar  6 17;26;39 comment by...Hyunsu -->
All DQMon alarm cleared.
<!-- Date: Thu Mar 6 16:46:51.467 2008 -->
Plug voltages trip and cannot be reset by the standard procedures. Willis comes ot fix it. 
<!-- Author: Joe Boudreau :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->
<!-- Comment: Thu Mar  6 17;54;40 comment by...Willis -->
I have no clue yet why plug CAEN crate 8 has been flaking
out today (like it decides not to flake out while I am here).
<!-- Date: Thu Mar 6 16:48:53.815 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259072>259072</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L1T 
CMU 
</PRE>
We did not successfully even start these runs.  We spent this part of the store just trying to understand the trigger rate troubles.  
<!-- Author: cdfscico -->


<!-- Date: Thu Mar 6 16:48:54.552 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259075>259075</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L1T 
CMU 
</PRE>
We did not successfully even start these runs.  We spent this part of the store just trying to understand the trigger rate troubles.  
<!-- Author: cdfscico -->




<!-- Date: Thu Mar 6 17:11:12.310 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1190&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1191  width='300' align='top'></a></td><td>Bias voltage has a small increase.  We set this to "ignore"</td></tr></table>
<!-- Author: Joe Boudreau -->


<!-- Date: Thu Mar 6 17:14:39.060 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1192&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1193  width='300' align='top'></a></td><td>And another SVX Bias voltage, also set to ignore.</td></tr></table>
<!-- Author: Joe Boudreau -->



<!-- Date: Thu Mar 6 17:33:54.836 2008 -->
L2 decision TO. Auto HRR fixed it.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->
<!-- Comment: Thu Mar  6 18;35;33 comment by...Hyunsu -->
Again
<!-- Date: Thu Mar 6 17:38:14.058 2008 -->
<table><tr><td><a href=/elog/notebooks/2008/Thu_Mar__6_2008_Eve.173747.ps align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1195  align='top'></a></td><td>CMP plot on fixlist seems fixed, CMP top stack 1 was turned off because it was oscillating. After the access, it now looks fine (neither hot nor dead). </td></tr></table>
<!-- Author: Sarah -->
<!-- Comment: Thu Mar  6 17;38;49 comment by...Sarah -->
note, you have to actually click on he plot to see the CMP Top plot .




<!-- Date: Thu Mar 6 18:08:14.818 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1196&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1197  width='300' align='top'></a></td><td>Another fixed problem, the hot channel in CXP-NW#4 is fixed.</td></tr></table>
<!-- Author: Sarah -->
<!-- Comment: Thu Mar  6 18;08;30 comment by...Sarah -->
that is, CSP NW #4

<!-- Date: Thu Mar 6 18:24:57.442 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259072>259072</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=259072 target=_runSummary>259072</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->



<!-- Date: Thu Mar 6 18:28:58.574 2008 -->
CEM went to yellow in the global alarm page. I saw couple of PMT went to red.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->
<!-- Comment: Thu Mar  6 18;31;50 comment by...Hyunsu -->
Couple of PMTs Phi 7-17, eta 0-9. Not whole of it. 
<!-- Date: Thu Mar 6 18:30:55.657 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1199&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1199  width='300' align='top'></a></td><td>This plot shows an inefficiency in COT08 Slot 5 and COT08 Slot 14.  I have updated COT timing offsets in the XTCD monitoring that should "fix" the apparent ineffiency in COT08 slot 14.  This fix will show up after starting a new run or restarting trigmon.  There is NO reason to restart the run or trigmon for this, but if either of these is done for another reason, please post the new plot after enough statistics.  thanks!</td></tr></table>
<!-- Author: Heather -->



<!-- Date: Thu Mar 6 18:34:26.541 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1200&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1200  width='300' align='top'></a></td><td>Red lit was seen.</td></tr></table>
<!-- Author: Hyunsu -->



<!-- Date: Thu Mar 6 18:47:37.830 2008 -->
Plug HV 11 was off. Check passed was off for 11. I tried reset but, it didn't come up. I am going to power cycling of this crate(Step2). 
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->
<!-- Comment: Thu Mar  6 18;54;17 comment by...Hyunsu -->
I turned it on after 5 minutes wait, and check passed red was back. I started monitoring program and looks everything ok.  

<!-- Date: Thu Mar 6 19:03:04.310 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259075>259075</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=259075 target=_runSummary>259075</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->



<!-- Date: Thu Mar 6 19:03:04.477 2008 -->
<table border=0><td><tr><td bgcolor=#e0e030>The CDF Solenoid is ramping down!!!</td></tr></td></table>
<!-- Author: Joe :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->


<!-- Date: Thu Mar 6 19:03:11.738 2008 -->
We had a lot of inhibit from B field COT TOF Plug HV PES VME Power.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078' target=_top>259078</a>) -->
<!-- Comment: Thu Mar  6 19;03;52 comment by...Hyunsu -->
I am going to End of Store(CLC ON).

<!-- Comment: Thu Mar  6 19;16;51 comment by...Hyunsu -->
East plug HV 7-12 were off and b0pcal06-b0pcal11 were red in the RC. 
<!-- Date: Thu Mar 6 19:08:01.438 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078>259078</A>
 Terminated at 2008.03.06 19:07:20
<!-- Author: RunControl -->


<!-- Date: Thu Mar 6 19:08:02.691 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078>259078</A>
TERMINATE: End of run because of solenoid trip
<!-- Author: Hyunsu -->



<!-- Date: Thu Mar 6 19:21:23.882 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259077>259077</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=259077 target=_runSummary>259077</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->



<!-- Date: Thu Mar 6 19:21:23.727 2008 -->
<pre>I have checked COT LVs and readings are fine.  
 
(In the past, LNE vicor power has been tripping when the magnetic  
field went down)</pre>
<!-- Author: Shin-Shan Yu (Eiko) -->


<!-- Date: Thu Mar 6 19:24:15.751 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1201&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1201  width='300' align='top'></a></td><td>WHA plot, looks a bit hot for latest run (259078)</td></tr></table>
<!-- Author: Sarah -->


<!-- Date: Thu Mar 6 19:24:28.074 2008 -->
<pre>Request from Eiko:  when we get the solenoid up, do a COT calibration.  </pre>
<!-- Author: Joe -->


<!-- Date: Thu Mar 6 19:37:44.353 2008 -->
<table border=0><td><tr><td bgcolor=#e0e030>Page Silicon before turning Silicon ON</td></tr></td></table>
<!-- Author: SashaS -->


<!-- Date: Thu Mar 6 19:38:25.280 2008 -->
<p>It appears that power is off to the east plug: the pcal crates 
are all red in PSM, the CAEN crates are all not responding, and 
the solenoid crashed as if the crash button powered by the east 
plug 60 Hz.  Since the front-end crates are dead, this indicates 
both 60 Hz and 400 Hz are off.  Oddly, in the RACKS display for 
the east plug, POWER PERMIT is off on EPSE but not EPNE.  I've  
just talked with Sebastian; we will try and get a hold of Dervin 
to see if he has any other ideas, and then probably try bringing 
up the east plug with the RACKS controls.</p>
<!-- Author: Steve Hahn -->


<!-- Date: Thu Mar 6 19:42:13.610 2008 -->
<pre>We will switch to cosmics now, to excercise this system until  
we get the solenoid up. </pre>
<!-- Author: Joe -->
<!-- Comment: Thu Mar  6 19;54;12 comment by...Joe -->
Bad idea.. 100% dead time.  We will try L2Torture
<!-- Date: Thu Mar 6 19:44:20.388 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259081>259081</A>
 Activated at 2008.03.06 19:43:38
<!-- Author: RunControl -->


<!-- Date: Thu Mar 6 19:44:21.173 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259081>259081</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,1019,706] 
<!-- Author: Hyunsu -->


<!-- Date: Thu Mar 6 19:46:31.405 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259077>259077</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
L1T 
CMU 
</PRE>
Still fixing L1/CMU problems. 
<!-- Author: cdfscico -->


<!-- Date: Thu Mar 6 19:47:21.885 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259078>259078</A>
cal: <tt>WHAPMT possibly bad (it looks hot, but is not up to the 12 consecutive hot channels level)</tt>
<!-- Author: Consumer Operator(Sarah Budd) -->



<!-- Date: Thu Mar 6 19:51:31.623 2008 -->
Scaler monitor error. High dead time (at 100%). I am not sure why it happened in the cosmic run. 
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259081' target=_top>259081</a>) -->
<!-- Comment: Thu Mar  6 19;59;33 comment by...Hyunsu -->
We restart run with L2Torture to wait recovery.
<!-- Comment: Thu Mar  6 20;04;23 comment by...Hyunsu -->
L2 torture also have same problem. We will run with AAA_shotsetup.
<!-- Date: Thu Mar 6 19:53:38.641 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259081>259081</A>
 Terminated at 2008.03.06 19:53:24
<!-- Author: RunControl -->


<!-- Date: Thu Mar 6 19:53:39.691 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259081>259081</A>
TERMINATE: done
<!-- Author: Hyunsu -->




<!-- Date: Thu Mar 6 19:55:38.229 2008 -->
<p>After talking with Dervin, we believe that there was a dual 
fan failure in the EPNE rack, there was already a single fan 
failure in the middle fanpack.  We have a agreement forged in the 
Peter Wilson/Willis Sakumoto days, that we can bypass fan alarms 
if this prevents us from data-taking; we still have temperature 
monitoring in the rack protection.  I've talked to Sebastian; 
I'm going downstairs to try and figure out how to do this with  
K.C.'s and Bill Noe's help.</p>
<!-- Author: Steve Hahn :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259081' target=_top>259081</a>) -->


<!-- Date: Thu Mar 6 19:59:48.377 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259082>259082</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [19,1021,705] 
<!-- Author: Hyunsu -->


<!-- Date: Thu Mar 6 20:03:45.937 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259082>259082</A>
TERMINATE: done
<!-- Author: Hyunsu -->



<!-- Date: Thu Mar 6 20:08:24.268 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259083>259083</A>
ACTIVATE: RunSet: AAA_SHOTSETUP  Trigger: PHYSICS_5_01 [4,1032,717] 
<!-- Author: Hyunsu -->



<!-- Date: Thu Mar 6 20:09:00.895 2008 -->
<pre>Experts decide that a fan alarm in a EPNE-MID triggered the shutdown 60 Hz.  The 60Hz shutdown triggered the Cryo crash. 
Steve+Willis are bypassing the alarm.  Then we may operate the 
cryostat. </pre>
<!-- Author: Joe :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259083' target=_top>259083</a>) -->


<!-- Date: Thu Mar 6 20:09:19.299 2008 -->
 
 Attention !!!. FERML_TM2_HEVB2B_EVENTS_PENDING_TIMEOUT !!!  
 - AUTO HRR will be issued. Auto HRR fixed it.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259083' target=_top>259083</a>) -->

<!-- Date: Thu Mar 6 20:18:20.590 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259083>259083</A>
TERMINATE: done to include pcal06-pcal11 (power was back)
<!-- Author: Hyunsu -->



<!-- Date: Thu Mar 6 20:18:52.848 2008 -->
Power is back.. Request ramp-up of the solenoid.
<!-- Author: Joe  -->

<!-- Date: Thu Mar 6 20:21:19.674 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259084>259084</A>
ACTIVATE: RunSet: AAA_SHOTSETUP  Trigger: PHYSICS_5_01 [4,1032,717] 
<!-- Author: Hyunsu -->



<!-- Date: Thu Mar 6 20:24:03.078 2008 -->
<p>With Bill Noe's guidance, I changed one of the fan inputs on 
the command block to TRUE, so that we no longer have a dual fan 
failure.  Resetting the east plug brought all power back.  K.C. 
is currently bringing the solenoid back up.</p>
<!-- Author: Steve Hahn :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259084' target=_top>259084</a>) -->


<!-- Date: Thu Mar 6 20:26:45.958 2008 -->
GAM and HBM were red in the global alarm page. 
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259084' target=_top>259084</a>) -->
<!-- Comment: Thu Mar  6 20;51;30 comment by...Hyunsu -->
Steve Hahn rebooted VNODE1 and had some work. Now it is ok.
<!-- Date: Thu Mar 6 20:28:49.252 2008 -->
<p>I increased the HV on the SE CEM Gamma supply from 1275.0 V 
to 1280.0 V after checking all the CEM HVs.  I would <b>not</b> 
mark the previous run bad, but I'll talk with Larry Nodulman  
about it tomorrow.</p>
<!-- Author: Steve Hahn :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259084' target=_top>259084</a>) -->



<!-- Date: Thu Mar 6 21:00:38.513 2008 -->
done TO from b0pcal07. It was red. I tried to save trace but, I found it was stopped. I started it and shepherded this crate. Now ok.  
<pre> 
b0pcal07-> i 
 
  NAME        ENTRY       TID    PRI   STATUS      PC       SP     ERRNO  DELAY 
---------- ------------ -------- --- ---------- -------- -------- ------- ----- 
tExcTask   excTask      1effb050   0 PEND         2e76c8 1effaf30       0     0 
tNetTask   netTask      1edb9010   1 PEND         2da0a0 1edb8f20       0     0 
tPortmapd  portmapd     1ed95810   1 PEND         2da0a0 1ed955b0  3d0002     0 
tShell     shell        1ed8a600   5 READY        2df9fc 1ed8a1c0  1c0001     0 
tRlogind   rlogind      1ed923c0  15 PEND         2da0a0 1ed91f80       0     0 
t1         VISIONserver 1ed7e220 100 PEND         2da0a0 1ed7e050       0     0 
t2         ROBINserver  1ed79060 100 PEND         2da0a0 1ed78e90       0     0 
rtlm_sessiortlm_session 1ebc02f0 189 PEND         2da0a0 1ebc0040       0     0 
Readout    FER_readout2 1eb757e0 199 PEND         2da0a0 1eb754a0  3d0002     0 
Messenger  FER_messenge 1ed56f20 200 SUSPEND      29d8d0 1ed56b00  3d0004     0 
Watchdog   FER_watchdog 1fffa850 200 PEND+T       2e76c8 1fffa710  3d0004    56 
rtlm_main  rtlm_main    1ed73ea0 220 PEND         2da0a0 1ed73cc0       0     0 
Mon_III    FER_monitorI 1ebac520 220 DELAY        2df320 1ebac460  1c0001    29 
tLogTask   logTask      1eff8450 250 PEND         2e76c8 1eff8340       0     0 
value = 0 = 0x0 
b0pcal07-> tt Messenger 
2e65c0 vxTaskEntry    +68 : 1e622000 () 
1e622000 FER_messenger  +568: 1e5ddbf8 () 
1e5ddbf8 FER_smartInitMS+2718: 1e6197dc () 
1e6197dc FER_monitorSender+7fc: TipcSrvMsgSend (1ebbb040, 1) 
22aa84 TipcSrvMsgSend +398: TipcConnMsgSend (1ebb92e0, 1ebbb040) 
1b173c TipcConnMsgSend+268: TipcBufMsgInsertRecursive (1ebb9338, 1ebbb040, 1ebb 
2e0, 1ed56c64, 1ed56c68) 
1c9860 TipcBufMsgInsertRecursive+1368: TipcBufMsgInsertRecursive (1ebb9338, 1eb 
2b60, 0, 0, 1ed56c34) 
1c8960 TipcBufMsgInsertRecursive+468: memcpy () 
value = 0 = 0x0 
</pre>
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259084' target=_top>259084</a>) -->

<!-- Date: Thu Mar 6 21:09:57.063 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259084>259084</A>
TERMINATE: solenoid is on
<!-- Author: Hyunsu -->


<!-- Date: Thu Mar 6 21:12:25.266 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259085>259085</A>
ACTIVATE: RunSet: COTCALIB_ACE 
<!-- Author: Hyunsu -->


<!-- Date: Thu Mar 6 21:12:39.171 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259050>259050</A>
CALIBRATION: <U><B>toftac</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#toftac target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Thu Mar  6 21;15;55 comment by...Sarah -->
this looks fine, but I'm not sure why we got this message now.
<!-- Date: Thu Mar 6 21:12:40.316 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259085>259085</A>
TERMINATE: done
<!-- Author: Hyunsu -->




<!-- Date: Thu Mar 6 21:19:03.018 2008 -->
Cryo said solenoid is back. I put COT standby and put on all HV without silicon and COT.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259085' target=_top>259085</a>) -->
<!-- Comment: Thu Mar  6 21;23;00 comment by...Hyunsu -->
BMU alarm from SE 144. I turned off this sector.


<!-- Comment: Thu Mar  6 21;32;58 comment by...Hyunsu -->
turn on silicon by permission from silicon expert. And turn on COT after 15 min waiting.
<!-- Date: Thu Mar 6 21:24:36.312 2008 -->
<pre>From cryo:  will boot S5 (for Steve & Willis). </pre>
<!-- Author: Joe :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->

<!-- Date: Thu Mar 6 21:29:04.016 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259085>259085</A>
CALIBRATION: <U><B>cot</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#cot target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->
<!-- Comment: Thu Mar  6 21;33;40 comment by...Sarah -->
calibration has been updated 8 of the last ten times, but this number should be less than 7. (from the calibration output 'Out of the last 10 calibrations, constants have been updated 8 times')
Otherwise things are allright. We are notifying the expert.

<!-- Comment: Thu Mar  6 21;37;14 comment by...Shin-Shan Yu (Eiko) -->
<pre>Thanks for letting me know. I've checked log file and didnt find 
any crate or slot that has been drifting constantly. This
 calibration is fine.</pre>
<!-- Date: Thu Mar 6 21:34:38.739 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086>259086</A>
 Activated at 2008.03.06 21:34:13
<!-- Author: RunControl -->


<!-- Date: Thu Mar 6 21:34:39.204 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086>259086</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_5_01 [4,1032,717] 
<!-- Author: Hyunsu -->




<!-- Date: Thu Mar 6 21:38:37.888 2008 -->
DQMon alarm for IMU but, it was disappered.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->
<!-- Comment: Thu Mar  6 21;44;14 comment by...Hyunsu -->
PHA alarm.
<!-- Date: Thu Mar 6 21:40:45.420 2008 -->
L2 decision TO.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->
<!-- Comment: Thu Mar  6 21;52;06 comment by...Hyunsu -->
Again


<!-- Date: Thu Mar 6 21:56:58.847 2008 -->
 Attention !!!. CER_SVXMON_HALT_RECOVER_RUN_ERROR !!!  
  
 Stuck Cellid S/B2/W4/L4/C7-13 .  
 Auto HRR fixed it.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->


<!-- Date: Thu Mar 6 22:09:25.492 2008 -->
L00 alarm. I let it to CO and check it. We got one of yellow channel.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->


<!-- Date: Thu Mar 6 22:12:21.708 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1203&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1203  width='300' align='top'></a></td><td>L00, with yellow (no data) ladder</td></tr></table>
<!-- Author: Sarah -->
<!-- Comment: Thu Mar  6 22;18;05 comment by...Hyunsu -->
Roberto(silicon MPC) said it was known problem. We just keep it on eye. 

<!-- Comment: Thu Mar  6 22;21;09 comment by...Silicon Expert -->
This is a known problematic ladder. She is slower than the others and it takes some minutes to read data correctly. Thanks.


<!-- Comment: Thu Mar  6 23;44;41 comment by...Hyunsu -->
It is still persistent.
<!-- Comment: Fri Mar  7 06;41;52 comment by...Mauro -->
<pre>It finally went back to green.</pre>
<!-- Date: Thu Mar 6 22:21:13.062 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1204&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1205  width='300' align='top'></a></td><td>Xtc Slots Efficiency canvas is still red.</td></tr></table>
<!-- Author: Sarah -->
<!-- Comment: Thu Mar  6 22;34;14 comment by...Heather -->
<pre>Thanks.  Cot08 Slot14 looks better compared to the plot earlier this shift.  There are a few slots with lower efficiency that we would like.   We are investigating COT08 Slot5 (it may need to be swapped) and COT09 Slot5.

The data can still be marked good.  Let's keep an eye on it and make sure these slots don't get worse. </pre>


<!-- Date: Thu Mar 6 22:47:38.427 2008 -->
b0pcal07 went to red. I saved trace file as 080306Trace1.trc. And shepherding this crate. After that I had one more HRR because active went to red. Now it is ok.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->


<!-- Date: Thu Mar 6 22:57:26.223 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1206&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1207  width='300' align='top'></a></td><td>New hot channel in CEMPMT canvas (eta 0, phi 0) </td></tr></table>
<!-- Author: Sarah -->


<!-- Date: Thu Mar 6 23:02:34.746 2008 -->
Icicle HB alarm, but procmon is ok. And I can see updating from solenoid2 PC. I just keep eye on it.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->
<!-- Comment: Thu Mar  6 23;25;20 comment by...Hyunsu -->
It was gone by itself.
<!-- Date: Thu Mar 6 23:02:39.947 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-1208&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-1209  width='300' align='top'></a></td><td>WHA PMT plot looks similar to previous run.</td></tr></table>
<!-- Author: Sarah -->


<!-- Date: Thu Mar 6 23:03:15.824 2008 -->
Done TO. for crate/s: b0pcal08.
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->


<!-- Date: Thu Mar 6 23:18:22.002 2008 -->
Plug HV 08 trip again. It looks like similar with early of my shift. I saw check passed was off for plug 08. When I try to stop HV program, it was on by itself. During start HV monitoring one more trip happen. After I reset it, it was recovered. CPU usage is normal when I check it. 
<!-- Author: Hyunsu :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=259086' target=_top>259086</a>) -->
<!-- Comment: Thu Mar  6 23;21;02 comment by...Hyunsu -->
Plug HV 08 trip again.

<!-- Comment: Thu Mar  6 23;24;37 comment by...Hyunsu -->
I reset it again and start taking data.



<!-- Date: Thu Mar 6 23:55:24.985 2008 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>RunNumber</TH>
<TH>DataType</TH>
<TH>PhysicsTable</TH>
<TH>Begin</TH>
<TH>End</TH>
<TH>LiveTime</TH>
<TH>L3Accepts</TH>
<TH>DelivLumi nb<SUP>-1</SUP></TH>
<TH>LiveLumi nb<SUP>-1</SUP></TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=259075 target=_runSummary>259075</A> x3F403</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [4,1032,717]</FONT></TD>
<TD ALIGN=CENTER>15:41:32</TD>
<TD ALIGN=CENTER>16:02:51</TD>
<TD ALIGN=CENTER>00:06:32</TD>
<TD ALIGN=RIGHT>      34,479</TD>   <TD ALIGN=RIGHT>    156.14</TD>
<TD ALIGN=RIGHT>     45.08</TD>
<TD ALIGN=CENTER>0</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=259077 target=_runSummary>259077</A> x3F405</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [4,1032,717]</FONT></TD>
<TD ALIGN=CENTER>16:16:51</TD>
<TD ALIGN=CENTER>16:21:06</TD>
<TD ALIGN=CENTER>00:04:12</TD>
<TD ALIGN=RIGHT>      15,889</TD>   <TD ALIGN=RIGHT>     31.48</TD>
<TD ALIGN=RIGHT>     27.19</TD>
<TD ALIGN=CENTER>0</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=259078 target=_runSummary>259078</A> x3F406</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [4,1032,717]</FONT></TD>
<TD ALIGN=CENTER>16:23:54</TD>
<TD ALIGN=CENTER>19:07:20</TD>
<TD ALIGN=CENTER>01:57:42</TD>
<TD ALIGN=RIGHT>     829,435</TD>   <TD ALIGN=RIGHT>    908.68</TD>
<TD ALIGN=RIGHT>    648.64</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=259083 target=_runSummary>259083</A> x3F40B</TD>
<TD>DAQTEST</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [4,1032,717]</FONT></TD>
<TD ALIGN=CENTER>20:07:04</TD>
<TD ALIGN=CENTER>20:17:45</TD>
<TD ALIGN=CENTER>00:10:14</TD>
<TD ALIGN=RIGHT>      25,265</TD>   <TD ALIGN=RIGHT>     46.79</TD>
<TD ALIGN=RIGHT>     42.98</TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=259084 target=_runSummary>259084</A> x3F40C</TD>
<TD>DAQTEST</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [4,1032,717]</FONT></TD>
<TD ALIGN=CENTER>20:20:52</TD>
<TD ALIGN=CENTER>21:09:00</TD>
<TD ALIGN=CENTER>00:45:41</TD>
<TD ALIGN=RIGHT>      57,042</TD>   <TD ALIGN=RIGHT>    196.32</TD>
<TD ALIGN=RIGHT>    183.91</TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=259086 target=_runSummary>259086</A> x3F40E</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [4,1032,717]</FONT></TD>
<TD ALIGN=CENTER>21:34:13</TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>02:00:20</TD>
<TD ALIGN=RIGHT>     624,622</TD>   <TD ALIGN=RIGHT>    481.14</TD>
<TD ALIGN=RIGHT>    410.48</TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER></TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD>Totals</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD>
<TD>23:55:01</TD>
<TD ALIGN=CENTER>05:04:43</TD>
<TD ALIGN=RIGHT>     1,586,732</TD> <TD ALIGN=RIGHT>  1,820.56</TD>
<TD ALIGN=RIGHT>  1,358.28</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD></TR>

</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Thu Mar 6 23:56:26.439 2008 -->
<font color=red>Shift Summary: </font><pre>Very eventful shift, unfortunately datataking was not the *main* 
event this evening. 
 
Shift started after a major headache with the Level 2 rate. The problem was muon TDC's that did not recover after the power cut. 
 
Then we had problems with plug HV, and very poor efficiency. While experts were working we got another problem: the solenoid started ramping down, caused by a power cut to the crash button, caused by a second fan failure in the EPNE rack.  Bypassed the 
alarm, ramped up the solenoid, and started taking data more or less smoothly at 9:30 (run 259086).   
 
 
 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   259078 259086
Delivered Luminosity   2167 nb-1  
Acquired Luminosity    1335 nb-1  
Efficiency             61.6%
   
</td></tr></table></pre>
<!-- Author: Joe Boudreau -->