<!-- Title: Day shift. Wed Jan 22, 2003 -->
<!-- SciCo: R.J. Tesarek -->
<!-- DAQAce: T. Nelson,H. Kobayashi -->
<!-- MonAce: &nbsp; -->
<!-- CO: &nbsp; -->
<!-- OpManager: &nbsp; -->
<!-- Notes: -->
<pre>Priorities for this shift: 
  1.  Recover the detector 
  2.  Idenfity problems caused by power outage 
  3.  Control kaos 
  4.  satisfy requests to work with detector.</pre>


<!-- Date: Wed Jan 22 08:19:40 2003 -->
W. Badgett and Franck are working on DAQ system to recover it.
<!-- Author: Hiro/Tim -->


<!-- Date: Wed Jan 22 08:21:06 2003 -->
Steve Hahn says we can expect anarchy on the monitoring side for a while, as there are server problems downstairs.
<!-- Author: Tim/Hiro -->


<!-- Date: Wed Jan 22 08:23:54 2003 -->
We cannot use L3 due to air condition work.
<!-- Author: Hiro/Tim -->


<!-- Date: Wed Jan 22 08:29:51 2003 -->
<br><b>Power outage recovery in progress...</b> 
<ul> 
  <li>Initial problems bringing up IFIX servers.</li> 
  <li>b0dau30 disk problems,  reboot in progress.</li> 
</ul>
<!-- Author: R.J. Tesarek -->


<!-- Date: Wed Jan 22 08:35:37 2003 -->
CYR guy is working on air conditioner at 3rd floor.  
<!-- Author: Hiro/Tim -->


<!-- Date: Wed Jan 22 08:38:26 2003 -->
<br> 
Tony Vaiciulis is also working on CS/L.<br> 
He said that disk9 was not initialized after power outage.<br>
<!-- Author: Hiro/Tim -->


<!-- Date: Wed Jan 22 08:41:27 2003 -->
<br><b>SciCo Power:</b> 
<br>Intrepid SciCo uncovers inefficient use of plug space in 
power strips under his desk.  Recovers an additional power outlet 
by moving a wall wart. 
<br>
<!-- Author: R.J. Tesarek -->


<!-- Date: Wed Jan 22 08:41:59 2003 -->
<p>Just so there is some record here of this:</p> 
<p /> 
<p>All alarm tests this morning 6:00-7:00 AM were successful--flammable gas and ODH.  We had to test the flammable gas shunt trips for both the collision hall and the assembly building, so we had to trip experimental power <b>twice</b>. After testing the collision hall flammable gas shunt trips, we reset everything; but after the assembly building test, we left everything off till after the power outage, so we only had to reset twice, not thrice.  Note this means that, unlike the power outage Monday, power will be brought back gently to the detector, since we have to reset the output breakers of the 400 Hz MGs.   All detector HV was turned off via iFIX or manually during this.</p>
<!-- Author: Steve Hahn -->


<!-- Date: Wed Jan 22 08:46:02 2003 -->
<br><b>Trigger room noise:</b> 
<br>Loud noise originating from trigger room.  S. Hahn,  
R.J. Tesarek track down the source to the water pipes.  Suspect 
air in the lines. 
<br>
<!-- Author: R.J. Tesarek -->


<!-- Date: Wed Jan 22 08:46:15 2003 -->
<p>Server CDF_S2 is back up and running, so alarms should be reported correctly to iFIX.  Of course, many systems are still off, either at the iFIX node level or the detector itself.  Alarms are rampant!  If in doubt, call the process systems tech or me; I will report back to the control room as things come up.</p>
<!-- Author: Steve Hahn -->


<!-- Date: Wed Jan 22 08:48:17 2003 -->
<br><b>L3</b> 
<br>Air conditioning work holding off bringing back L3. 
<br>
<!-- Author: R.J. Tesarek -->
<!-- Comment: Wed Jan 22 09;54;43 comment by...Hiro/Tim -->
Air conditioned is completely off in third floor. Hopefully, we will not have a power trip (L3 farm is off, but online machines are on).
<!-- Date: Wed Jan 22 09:01:52 2003 -->
<br><b>Detector Power:</b> 
<br>Detector power returned.  D. Allen, S. Hahn 
<br>
<!-- Author: R.J. Tesarek -->


<!-- Date: Wed Jan 22 09:10:17 2003 -->
<pre>Have powered up all 1st and 2nd floor counting  
room racks and crates.    
 
Crates not working:  b0l1gl00 (Fred) and b0l2de00 
 
</pre>
<!-- Author: W.Badgett -->
<!-- Comment: Wed Jan 22 09;21;51 comment by...G. Feild -->
<pre>Popped out and reinserted the PFRED and FRED cards
and restored VME in the "Fred" crate.  It should work now.

</pre>
<!-- Comment: Wed Jan 22 09;38;42 comment by...Greg Feild -->
<pre>Recovered VME in the Global Level 2 decision crate.
Had to reseat ALL the cards including the crate CPU.</pre>
<!-- Date: Wed Jan 22 09:15:21 2003 -->
<br><b>Detector Power:</b> 
<br>Power resturns to detector.  Identified the following  
problems on recovery: 
<ul> 
 <li>Single fan failure for EPNW (top fan).</li> 
 <li>SVX racks having trouble with power up.</li> 
</ul>  
<!-- Author: R.J. Tesarek -->



<!-- Date: Wed Jan 22 09:36:58 2003 -->
Power supply troubles as of now: 
 
Endwall-W: CONWT_3,CSP_W alarming 
Endwall-E: CSP_E, CONEB_3 alarming; CONET_4 masked 
 
All SVX and BSU alarming.
<!-- Author: Tim -->



<!-- Date: Wed Jan 22 09:46:58 2003 -->
<pre>Restarted the SoftEvb proxy, as it seemed to  
have automatically restarted before the  
cdfonprd database was available. 
 
Also restarted DBbroker, which did not to  
seem to start up at boot time. 
</pre>
<!-- Author: W.Badgett :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157000' target=_top>157000</a>) -->



<!-- Date: Wed Jan 22 09:59:26 2003 -->
<br><b>COT Readout Crates:</b> 
<br> Dave Ambrose and Bob Wagner take the COT crates for front 
end work. 
<br>
<!-- Author: R.J. Tesarek -->


<!-- Date: Wed Jan 22 10:05:36 2003 -->
SVX power is up.
<!-- Author: Tim -->


<!-- Date: Wed Jan 22 10:12:00 2003 -->
Took a run with a few crates : all ccal, L1, L2 trigger, pcal crates, with software event builder.  Having problems with L2 decision crate.
<!-- Author: J. Nachtman :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157004' target=_top>157004</a>) -->


<!-- Date: Wed Jan 22 10:19:29 2003 -->
CSL powered up and all processes are running. 
There was a disk failure on startup. It was replaced 
with a spare.  
<!-- Author: Tony Vaiciulis -->


<!-- Date: Wed Jan 22 10:19:44 2003 -->
Problems in ColdStart with XFT00, XFT02, XFT04: 
 
(MLE) b0xft00:Messenger:10:14:35 AM->slot 4, FlashLabel comparison mismatch, hdwdb: xft_inp13v3.exo, board: xo (code 7) 
(MLE) b0xft02:Messenger:10:14:35 AM->slot 4, FlashLabel comparison mismatch, hdwdb: xft_inp13v3.exo, board: xo (code 7) 
(MLE) b0xft04:Messenger:10:14:35 AM->slot 4, FlashLabel comparison mismatch, hdwdb: xft_inp13v3.exo, board: xo (code 7)  
<!-- Author: J. Nachtman, F. Chlebana :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157007' target=_top>157007</a>) -->


<!-- Date: Wed Jan 22 10:25:20 2003 -->
Turning CMX 16E and 17E on. 
<!-- Author: Phil Schlabach -->


<!-- Date: Wed Jan 22 10:28:36 2003 -->
removed svt crates from the run due to error messages in svt06.  SVT06 is in error state.
<!-- Author: Jane and Frank -->

<!-- Date: Wed Jan 22 10:36:55 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157011">157011</A>
ACTIVATE: Start L2TORTURE run w/o xft02, xft04 and svt06 .
<!-- Author: Hirokazu 2080 -->
<!-- Comment: Wed Jan 22 10;45;05 comment by...Hiro -->
SVT expert knew svt06 problem.

<!-- Date: Wed Jan 22 10:40:15 2003 -->
<br><b>SVX rack power up:</b> 
<br>We experienced problems during the initial power up for 
the SVX racks.  It turns out that the Errors in FORMATION need  
to be cleared prior to resetting racks. 
<br>  
<!-- Author: R.J. Tesarek -->


<!-- Date: Wed Jan 22 10:43:23 2003 -->
CSP W power supplies being worked on.  Power Supply Monitoring on 3rd floor (for all supplies) will display stale information until work is complete.
<!-- Author: Tim -->


<!-- Date: Wed Jan 22 10:44:20 2003 -->
<table border=0><td><tr><td bgcolor=#e0e030><br><h2>b0cot00:</h2> 
<b>Dervin Allen and Roberto Davila found that the power 
supply for b0cot00 was manually turned off.  This crate will  
remain off until the person responsible fesses up.</b> 
<br></td></tr></td></table>
<!-- Author: Dervin Allen -->


<!-- Date: Wed Jan 22 10:52:35 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157011">157011</A>
TERMINATE: End run.
<!-- Author: Hirokazu 2080 -->
<!-- Comment: Wed Jan 22 10;56;29 comment by...Hiro -->
I took out PCAL06-11 due to laser run for experts. 
<!-- Date: Wed Jan 22 10:52:35 2003 -->
<br> 
HVAC maintenance person estimates a couple of more hours of work<br> 
in third floor computing room. Level 3 will not be turned on <br>until he is finished. In addition, b0dau36 (online development)<br> 
has been turned off to reduce heat load in room.
<!-- Author: jj -->


<!-- Date: Wed Jan 22 10:54:22 2003 -->
<br><br>Airconditioning Update:</b> 
<br>Work continues on the 3rd floor air conditioning.  The temperature is currently 84F and slowly rising.  Tony Viaculis 
is concerned about the Consumer Server Logger and if the temperature continues to rise he will shut it down.<br>
<!-- Author: R.J. Tesarek -->
<!-- Comment: Wed Jan 22 11;17;02 comment by...Guillelmo -->
<table border=0><td><tr><td bgcolor=#e0e030>HVAC maintenance person doesn't know how many hours <br> air-conditioned will be off, and the temperature is <br> 
higher and higher. So, I have stopped completely L3 <br> 
farm (and b0eb10). <p>
We will bring up L3/Evb a.s.a.p.</td></tr></td></table>
<!-- Comment: Wed Jan 22 11;18;10 comment by...jj -->
<pre>Guillelmo has turned off convertor nodes for Level 3 to reduce
heat load. Thermometer has been moved to the wall left of the
HVAC unit since this is where the over-temp trip is located.
In this location, thermometer reads 81F.</pre>
<!-- Date: Wed Jan 22 10:58:35 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157013">157013</A>
ACTIVATE: Start L2TORTURE run again.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 11:01:53 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157013">157013</A>
TERMINATE: End run b0tsi00 problem.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 11:03:14 2003 -->
<pre>The PEI power supply in b0wcal07 cannot have its +5 and -5 volt 
analog voltages adjusted over the full range expected.  I'm going 
to have a look at current draws in this and other wcal crates. 
I don't expect to have to turn off any crates though...</pre>
<!-- Author: M Jones -->
<!-- Comment: Wed Jan 22 11;53;55 comment by...M Jones -->
<pre>Okay... I measured current and voltages in b0wcal07 and b0wcal02.
wcal07:  vcc5 =  4.75V at crate, 13.3 A
         vee5 = -5.05V at crate, 15.7 A
wcal02:  vcc5 =  4.78V at crate, 5.00 V at PS, 14 A
         vee5 = -5.18V at crate, 5.40 V at PS, 15.8 A
I should go back and re-measure wcal07...  the current meter
might not have been zeroed when I made the measurements but so
far wcal07 doesn't look any different from wcal02.  The load
of these crates is nowhere near the 20 A rating for the supply.
I don't have any reason to suspect that any particular card in
the crate is causing an anomalously low voltage at the crate.</pre>
<!-- Date: Wed Jan 22 11:06:49 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157014">157014</A>
ACTIVATE: Start L2TORTURE RUN after tsi00 was stuck at HRR.
<!-- Author: Hirokazu 2080 -->




<!-- Date: Wed Jan 22 11:27:06 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157014">157014</A>
TERMINATE: End the run to take out WCAL01,02.
<!-- Author: Hirokazu 2080 -->
<!-- Comment: Wed Jan 22 11;29;44 comment by...Hiro -->
And also I took out CCAL12.

<!-- Date: Wed Jan 22 11:34:15 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157017">157017</A>
ACTIVATE: Start L2TORTURE RUN again
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 11:38:26 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157017">157017</A>
TERMINATE: End the run.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 11:46:10 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157019">157019</A>
ACTIVATE: Start run with SUMET5 talble.
<!-- Author: Hirokazu 2080 -->




<!-- Date: Wed Jan 22 11:55:39 2003 -->
We've reconnected the LV and signal cables to the SE wedges 16 and 17. HV is on to test it out: 16E is attached to 14E, 17E to 16/17E. Voltages are nominal+100V. There is some wierd behavior in either HVmon or the hardware which is causing it to turn yellow. I think it's because I've put the set values very close to the hardwired limit. What's funny is that the actuals are close to where they should be, the set points are reported high, thus putting the channels into an false undervoltage state. It will all be fine when I drop back to nominal.
<!-- Author: Phil  Schlabach, Sal Rappoccio -->


<!-- Date: Wed Jan 22 11:57:21 2003 -->
<pre>We also harvested 2 ASDs from the NE CMX crate.</pre>
<!-- Author: Phil Schlabach -->


<!-- Date: Wed Jan 22 12:02:57 2003 -->
<br><b>XFT Status:</b> 
<br>We have had trouble with the XFT today.  After paging the 
expert, we learn that the XFT was left in a standalone state 
by the expert yesterday evening.  The system will be returned to 
the standard configuration for checkout in a few minutes. 
<br>
<!-- Author: R.J. Tesarek -->
<!-- Comment: Wed Jan 22 13;05;53 comment by...Cheng-Ju -->
The XFT is now back in nominal configuration.  Aces have included
in RC successfully.
<!-- Date: Wed Jan 22 12:05:43 2003 -->
Done timeout from CCAL02 due to noresponding slot(slot4) in SMXR. 
 
<!-- Author: Hiro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157019' target=_top>157019</a>) -->


<!-- Date: Wed Jan 22 12:08:37 2003 -->
<pre>Status of online monitors:   
All the monitors are reset to run with online production
database.  After the power outage, needed to mount
/www area by hand on b0dap32 in order to get the 
consumer statemager to write the status.  </pre>
<!-- Author: kaori -->


<!-- Date: Wed Jan 22 12:17:28 2003 -->
Replaced bad zener diodes on transition boards in slots 19,20,21 of b0ccal07. Will resume work later in the evening on other ccal and pcal crates.
<!-- Author: Mark Mattson and Vivek Tiwari -->
<!-- Comment: Wed Jan 22 12;25;01 comment by...R.J. Tesarek -->
The work by Vivek and Mark were likely responsible for the problems observed earlier with b0ccal07.  We will attempt to
return this crate to the run at the next opportunity.
<!-- Date: Wed Jan 22 12:23:10 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157019">157019</A>
TERMINATE: End the run.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 12:41:17 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157022">157022</A>
ACTIVATE: Start SUMET5 run again.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 12:48:25 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157022">157022</A>
TERMINATE: End run .
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 12:53:29 2003 -->
<pre>Reverted DBbroker back to the production  
database (forgot to do that yesterday) -- which  
is why it crashed an hour or two ago when  
the integration database machine was taken  
down due to A/C work. 
</pre>
<!-- Author: W.Badgett -->

<!-- Date: Wed Jan 22 12:54:23 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157024">157024</A>
ACTIVATE: Start SUMET5 run with XFT00,02,04 crates.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 12:58:43 2003 -->
We got done timeout from CCAL02  3 times due to the no-responding slot(slot 4) in SMXR.
<!-- Author: Hiro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157024' target=_top>157024</a>) -->


<!-- Date: Wed Jan 22 13:11:44 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157024">157024</A>
TERMINATE: End the run due to expert request.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 13:22:06 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157029">157029</A>
ACTIVATE: Start SUMET5[3,231,318] run again.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 13:26:45 2003 -->
Done timeout from CCAL02 due to non-responsing slot in SMXR.
<!-- Author: Hiro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157029' target=_top>157029</a>) -->


<!-- Date: Wed Jan 22 13:31:01 2003 -->
<pre>PSM is running again</pre>
<!-- Author: M Wolter -->

<!-- Date: Wed Jan 22 13:41:05 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157034">157034</A>
ACTIVATE: Start SUMET5[3,231,318] run again.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 13:43:48 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157034">157034</A>
TERMINATE: End run.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 13:46:20 2003 -->
<br><b>E. Plug power:</b> 
<br>E. Plug power was cut due to a drip sensor fault for NE 
racks.  Roberto Davila found dust and debris on the drip sensor. 
The sensor has been cleared and we are awaiting the plug HV  
folk to power down the CAEN system on the East plug before  
restoring power. 
<br>
<!-- Author: R.J. Tesarek -->

<!-- Date: Wed Jan 22 13:46:35 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157035">157035</A>
ACTIVATE: Start new SUMET5 run due to the expert request.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 13:54:58 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157035">157035</A>
TERMINATE: End run
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 13:59:12 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157036">157036</A>
ACTIVATE: SMXR calibration w/o East plug.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 13:59:12 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157036">157036</A>
TERMINATE: End SMXR calibration w/o East plug.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:07:08 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157037">157037</A>
ACTIVATE: Start CAL QIE calibration w/o East plug(PCAL06-11) and WCAL07.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:07:08 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157037">157037</A>
TERMINATE: End CAL QIE calibration.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:09:38 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157039">157039</A>
ACTIVATE: Start CLC QIE calibration.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:09:38 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157039">157039</A>
TERMINATE: End CLC QIE calibration.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:11:47 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157043">157043</A>
ACTIVATE: Start BSC QIE calibration
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:12:28 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157043">157043</A>
TERMINATE: End BSC QIE calibration.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 14:14:22 2003 -->
I made a short test run with the XTRP and Matchbox crates to track down a problem with one of the CMX bits.
<!-- Author: Jim Kraus :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157033' target=_top>157033</a>) -->

<!-- Date: Wed Jan 22 14:15:30 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157045">157045</A>
ACTIVATE: Start TOF QIE calibration w/o WCAL07.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:15:30 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157045">157045</A>
TERMINATE: End TOF QIE calibration w/o WCAL07.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:28:10 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157051">157051</A>
ACTIVATE: Start SMXR QIE calibration with all crate.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:28:28 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157051">157051</A>
TERMINATE: End SMXR calibration with all crate.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 14:28:56 2003 -->
<br><b>E. Plug power:</b> 
<br>Power returned to the East plug.  Stefano Zucchelli 
is checking out the plug HV.  Karen Byrum is looking at SMX 
calibrations. 
<br>
<!-- Author: R.J. Tesarek -->


<!-- Date: Wed Jan 22 14:31:08 2003 -->
<pre>Worked on Isolist energy errors.  Immediately saw 
channel 0 always sending data on the input to Isolist. 
Reseating the channel 0 fiber further up the chain  
at IsoClique made it go away.   Errors remained however,  
until we power cycled and reseated IsoClique.  Then 
the errors were much reduced,  from 10% to &lt;1%.  We 
will need an hour-long spike run with L3 to reestablish 
the baseline error rate.  </pre>
<!-- Author: Steve Kuhlmann, Bob Blair -->


<!-- Date: Wed Jan 22 14:36:05 2003 -->
<pre>A problem with the gains in towers 2 & 3 of CEM 22E has been  
noted.  Larry Nodulman sees it in left/right ratios.  Looking at 
LED runs I see 2L22E takes a large downward step between 
runs 155384 and 155547 while 3L22E took a large upward step.  I 
checked the high voltage reading on the tubes and it has been 
unchanged on towers 2 & 3 for better than 1 year.  Looked at 
signal and high voltage cables on tubes 2L22E and 3L22E.  The 
labels indicate the signal cables are correct.  Also true of 
labels on high voltage cables.  So I am confused as to the cause 
of the problem at this point.</pre>
<!-- Author: Bob Wagner (ANL) -->
<!-- Comment: Wed Jan 22 14;42;48 comment by...Bob Wagner (ANL) -->
<pre>By the way, the jump corresponds to the time between 02:00 on
15 Dec 2002 and 02:00 18 Dec 2002.  There was a supervised access
during this period.</pre>
<!-- Comment: Wed Jan 22 14;55;10 comment by...Larry -->
Actually Kostas Kordas found this validating offline
<!-- Date: Wed Jan 22 14:40:16 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157055">157055</A>
ACTIVATE: Start CAL QIE calibration w/o WCAL07
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:40:16 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157055">157055</A>
TERMINATE: End CAL QIE calibration.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:41:46 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157056">157056</A>
ACTIVATE: Start CAL QIE calibration with default crate.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:42:45 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157056">157056</A>
TERMINATE: End CAL QIE calibration with default crate.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 14:44:10 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157058">157058</A>
ACTIVATE: Start TOF QIE calibration with default crate.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:44:55 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157058">157058</A>
TERMINATE: End TOF QIE calibration.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 14:46:43 2003 -->
<pre>Borrowing the trigger to test a new DPS table. 
Note: SVT06 still doesn't work.</pre>
<!-- Author: Greg F. -->


<!-- Date: Wed Jan 22 14:47:50 2003 -->
<pre>Have set the flag IgnoreState=false for  
the CSL in the hardware database so that  
Run Control will do returned-state  
checking for CSL. 
 
Ben K. thinks it should be OK. 
 
</pre>
<!-- Author: W.Badgett :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157059' target=_top>157059</a>) -->

<!-- Date: Wed Jan 22 14:50:30 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157060">157060</A>
ACTIVATE: test dps test table
<!-- Author: feild -->


<!-- Date: Wed Jan 22 14:54:24 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157061">157061</A>
ACTIVATE: Start CAL QIE calibration again because previous run didn't show up DB.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 14:55:09 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157061">157061</A>
TERMINATE: End CAL QIE calibration.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 14:55:39 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157062">157062</A>
ACTIVATE: this time with the right table
<!-- Author: feild -->



<!-- Date: Wed Jan 22 14:59:58 2003 -->
Isamu Nakamura took CCAL01 crate.
<!-- Author: Hirokazu -->


<!-- Date: Wed Jan 22 15:04:07 2003 -->
I took CAL QIE calibration twice. 
But the run didn't show up in DB.  
<!-- Author: Hiro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157061' target=_top>157061</a>) -->


<!-- Date: Wed Jan 22 15:07:36 2003 -->
<pre>COT04 : Third time (or tracer) is the charm. I was able to run in calib cont mode with a very high rate. I have changed the TDC readout mode to 0 (from 2) This crate needs to be tested further in normal data taking. 
</pre>
<!-- Author: dan -->
<!-- Comment: Wed Jan 22 15;40;11 comment by...dan -->
Tracer # 135 (installed yesterday) was pulled and #123, the Fermi Id  #for #123 is 562639

<!-- Comment: Wed Jan 22 15;41;22 comment by...dan -->
<pre>Note that #135 had corrected the data corruption problem -but introduced bunch counter mismatches. #123 has not yet given bunch crossing mismatches, but Frank please test for VRB data corruption....
</pre>
<!-- Date: Wed Jan 22 15:12:13 2003 -->
<pre>Releasing the trigger for a while.</pre>
<!-- Author: Greg F. -->


<!-- Date: Wed Jan 22 15:16:42 2003 -->
<pre>To further verify that the CEM 22E signal cables are plugged into 
the correct tube, I unplugged successively 3L22E and 2L22E signal 
cables and terminated them in 50 ohms.  Rick Tesarek then  
verified that the pedestal for the correct channel read zero. 
We observed both signal cables are correctly mapped into the 
proper ADMEM locations.</pre>
<!-- Author: Bob Wagner (ANL) -->


<!-- Date: Wed Jan 22 15:17:59 2003 -->
<br><b>Air conditioning:</b> 
<br> The airconditioning on the 3rd floor was recovered about 
1-1/2 hrs ago.  L3 is being brought up now. 
<br>
<!-- Author: R.J. Tesarek -->

<!-- Date: Wed Jan 22 15:20:43 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157067">157067</A>
 Activated at 2003.01.22 15:20:37
<!-- Author: RunControl -->


<!-- Date: Wed Jan 22 15:22:09 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157067">157067</A>
ACTIVATE: Start cosmics run to exercise DAQ system.
<!-- Author: Hirokazu 2080 -->
<!-- Comment: Wed Jan 22 15;24;34 comment by...Hiro -->
CMX01,HTDC00,IMU00,MUSC00 and MUTR00 didn't pass coldstart. 


<!-- Date: Wed Jan 22 15:25:05 2003 -->
<br><b>DB Work:</b> 
<br>During the airconditioning work, the online integration 
database was brought down.  The DB experts were not notified 
when this happened and would like to be should this happen in 
the future. 
<br>
<!-- Author: R.J. Tesarek -->

<!-- Date: Wed Jan 22 15:29:18 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157067">157067</A>
 Terminated at 2003.01.22 15:29:16
<!-- Author: RunControl -->


<!-- Date: Wed Jan 22 15:29:30 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157067">157067</A>
TERMINATE: End cosmics run.
<!-- Author: Hirokazu 2080 -->



<!-- Date: Wed Jan 22 15:35:23 2003 -->
<pre>IMU Access Report 
 
Started re-installing southwest BSU/TSU CCU's.  Not finished.</pre>
<!-- Author: Camille Ginsburg and Dan Cyr -->

<!-- Date: Wed Jan 22 15:38:33 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157070">157070</A>
 Activated at 2003.01.22 15:38:09
<!-- Author: RunControl -->


<!-- Date: Wed Jan 22 15:39:09 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157070">157070</A>
 Terminated at 2003.01.22 15:38:49
<!-- Author: RunControl -->



<!-- Date: Wed Jan 22 15:41:22 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157072">157072</A>
ACTIVATE: testing b0ccal01
<!-- Author: Andrew K x2080 -->



<!-- Date: Wed Jan 22 15:42:13 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157071">157071</A>
 Activated at 2003.01.22 15:41:48
<!-- Author: RunControl -->


<!-- Date: Wed Jan 22 15:42:13 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157071">157071</A>
ACTIVATE: Start cosmics run to exercise DAQ system.
<!-- Author: Hirokazu 2080 -->


<!-- Date: Wed Jan 22 15:45:14 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157072">157072</A>
TERMINATE: failure

<!-- Author: Andrew K x2080 -->



<!-- Date: Wed Jan 22 15:45:14 2003 -->
From the scan with the scope we did today, it looks like we fixed 9 CMX channels on Saturday in Wedges 15E & 16E. I am leaving the HV to this region on.
<!-- Author: Phil Schlabach -->

<!-- Date: Wed Jan 22 15:49:35 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157074">157074</A>
ACTIVATE: test

<!-- Author: Andrew K x2080 -->


<!-- Date: Wed Jan 22 15:50:20 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157074">157074</A>
TERMINATE: failed

<!-- Author: Andrew K x2080 -->


<!-- Date: Wed Jan 22 15:51:28 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157075">157075</A>
ACTIVATE: test

<!-- Author: Andrew K x2080 -->


<!-- Date: Wed Jan 22 15:51:28 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157075">157075</A>
TERMINATE: fail
<!-- Author: Andrew K x2080 -->


<!-- Date: Wed Jan 22 15:55:25 2003 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>15:55:01</TD>                                                       
<TD ALIGN=CENTER>::</TD>                                                
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->


<!-- Date: Wed Jan 22 15:57:07 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157076">157076</A>
ACTIVATE: test ccal01

<!-- Author: Andrew K x2080 -->


<!-- Date: Wed Jan 22 15:58:16 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=157076">157076</A>
TERMINATE: success?
<!-- Author: Andrew K x2080 -->



<!-- Date: Wed Jan 22 15:58:50 2003 -->
<font color=red>Shift Summary: </font><pre>Power Outage Recovery: 
 
Problems with systems during recovery (new problems): 
  o Single Fan Failure (EPNW, top fan) 
  o SVX-FIB racks have trouble coming up (need secret decoder ring from Rob). 
  o Airconditioning on the 3rd floor fails. 
    + L3 down 
    + Online development DB down 
    + Online integration DB down 
    + b0dap30 down. 
  o b0ccal07,b0ccal12 problems. 
  o Dijet trigger rates 6kHz.  
  o Problems with CSP power supply. 
 
Detector work: 
  o COT readout work. 
  o TOF PMT replacement. 
  o CHA PMT work. 
  o Protection diode replacement slots 19,20,21 of b0ccal07. 
  o Isolation board work for the trigger. 
  o CCU work. 
  o Hadron ASD work. 
  o EM timing work. 
  o work on CEM PMT wedge 22W (tower 9). 
  o XEF work for b0ccal01 
  o b0cot04 work appears to be successful. 
 
Current problems with detector. 
  o b0cot00 was turned off without notifying shift crew. 
  o b0ccal02, SMXR in slot=4 giving done time outs. 
  o SVT06 is not working 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   
Delivered Luminosity   0  
Acquired Luminosity    0  
Efficiency             -
   
</td></tr></table></pre>
<!-- Author: R.J. Tesarek -->