<!-- Title: Day shift. Tue Jun  8, 2004 -->
<!-- SciCo: AB Wicklund -->
<!-- DAQAce: C. Doerr, M. Rossi -->
<!-- MonAce: M. Harr, J. Yamaoka -->
<!-- CO: W. Badgett -->
<!-- OpManager: JJ -->
<!-- Notes: -->
<pre>Taking physics data, store 3562, Lumi= 25 10**30, R183786</pre>


<!-- Date: Tue Jun 8 08:32:20.271 2004 -->
<pre>We had a Consumer Error: 
 
Attention !!!. CER_SVXMON_HALT_RECOVER_RUN_ERROR !!!  
Stuck Cellid I/B5/W5/L1/C4-7 .  
 
HRR helped.</pre>
<!-- Author: Melisa :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183786' target=_top>183786</a>) -->


<!-- Date: Tue Jun 8 09:19:58.631 2004 -->
<pre>halted run because of high abort gap losses.</pre>
<!-- Author: Christian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183786' target=_top>183786</a>) -->
<!-- Comment: Tue Jun  8 09;24;15 comment by...abw -->
<pre>SVX-> sby.  RF and L1Col both died giving huge glitch in 
PAGC.</pre>


<!-- Date: Tue Jun 8 09:26:10.283 2004 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2004&action=view&page=-8539&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2004&action=view&page=-8539  width='300' align='top'></a></td><td>Noticed that the Level 2 trigger, bit 92,  
name <A HREF=/java/cdfdb/servlet/TriggerDump?RUN=183786&BIT=92&TRIGGERNAME=L2_TEST_B_CHARM_PHI|[2]&LEVEL=2> 
L2_TEST_B_CHARM_PHI [2]</A> was way off from expected in XMon, with a zScore of +181.   Looks like someone put in the  
expected cross section in the trigger database, but not  
the trigger rate parameterization cofficients.  (See link).   Perhaps one of the Trigger Gurus could set  
these values?</td></tr></table>
<!-- Author: W.Badgett -->


<!-- Date: Tue Jun 8 09:31:28.770 2004 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2004&action=view&page=-8540&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2004&action=view&page=-8540  width='300' align='top'></a></td><td>Was wondering if the non-diagonal elements on the  
"XTRPMon: Track Trig bits != TT Sim != prefred" plot to the  
left are important.  In particular, x bin 6 has a  
large non-diagonal occupancy.  It has been this way  
for some time, but does not match the C.O.  
<A HREF=/internal/mon/cdfdoc/Consumers/TrigMon/Slides/trigmon_rightframe.html#xtrpmonsummary> 
reference plot</A>.   Perhaps an XTRP expert could  
comment? 
</td></tr></table>
<!-- Author: W.Badgett -->
<!-- Comment: Tue Jun  8 11;04;17 comment by...KPitts -->
12 errors out of 22,600 events is annoying, but not a problem.  These are errors reading the data out of the 2-track board, they are not errors in the trigger path.  These readout errors all go away when we put in the 3-track board.   The errors that bother me are the DIRAC bits in wedges 17 and 18.  We will check on these.
<!-- Date: Tue Jun 8 09:36:39.295 2004 -->
<pre>Glitch was caused by ComEd 345 kV line glitch. 
MCR reports that TEL tripped twice and expert  
is looking at it. We remain with svx on sby.</pre>
<!-- Author: abw -->


<!-- Date: Tue Jun 8 09:45:11.339 2004 -->
COT cold trap alarm. Cryo has to put in 2nd 
cold trap. Should have no effect on COT.
<!-- Author: abw -->


<!-- Date: Tue Jun 8 09:48:19.349 2004 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2004&action=view&page=-8541&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2004&action=view&page=-8541  width='300' align='top'></a></td><td>There was a trip for some HV in the tevitron that took down the electron lens and it looks like 1 of the RF. We halted the run and put the silicon in standby. Still waiting for the abort to clear out. </td></tr></table>
<!-- Author: Matt -->


<!-- Date: Tue Jun 8 09:54:24.091 2004 -->
<pre>Cryo calls - they have to replace a fan in B0, 
AC in coll. hall will be off for ~4 hours.</pre>
<!-- Author: abw -->


<!-- Date: Tue Jun 8 09:55:17.934 2004 -->
Steve from MCR called 10 minutes ago and asked if we wanted to 
scrape abort gaps. As we spoke, the electron lens tripped again 
so we will wait for a period of stability before deciding what 
to do.  
<!-- Author: jj -->


<!-- Date: Tue Jun 8 10:05:04.917 2004 -->
Talked to Steve again. MCR will move collimators and try to  
get abort gap losses down. <br> 
I also informed him we have HVAC off in Collision Hall for 
an hour or two.
<!-- Author: JJ -->
<!-- Comment: Tue Jun  8 10;05;21 comment by...jj -->
Steve in MCR...


<!-- Date: Tue Jun 8 10:21:25.452 2004 -->
<pre>MCR turned off TeV longitudinal damper to reduce abort 
gap losses.  They have gotten PAGC to ~19, we 
will turn on HV again.</pre>
<!-- Author: abw -->
<!-- Comment: Tue Jun  8 10;25;26 comment by...abw -->
<pre>PAGC losses appear to be rising again- hold off on 
turning back on</pre>


<!-- Date: Tue Jun 8 10:26:59.101 2004 -->
Abort gap losses still not acceptable. I talked to MCR Steve 
and he will continue to nudge collimators.
<!-- Author: jj -->


<!-- Date: Tue Jun 8 10:38:11.512 2004 -->
<pre>CMP Bottom D4 tripped off while we were in standby.  Two attempts to reset to standby have failed.  Called expert.</pre>
<!-- Author: Jared -->


<!-- Date: Tue Jun 8 10:38:17.899 2004 -->
CMP (BD4) trips while on standby during 
scraping. Paged muon experts.
<!-- Author: abw -->


<!-- Date: Tue Jun 8 10:38:53.475 2004 -->
<pre>Currents in the AVDD has all gone up slightly in coincedence with the RF trip in the accelerator.  
 
Also there was a trip in 
 
TRIP: ISL B5 W0 Ext Trip 
TRIP: ISL B5 W0 2VDOIM Over Current 
 
This trip was NOT coincedent with the RF trip.</pre>
<!-- Author: Matt :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183786' target=_top>183786</a>) -->


<!-- Date: Tue Jun 8 10:56:47.594 2004 -->
<pre>went from Halt to Standby in RC after request from  
Silion expert (David).</pre>
<!-- Author: Christian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183786' target=_top>183786</a>) -->
<!-- Comment: Tue Jun  8 10;57;27 comment by...Christian -->
<pre>....ooops, what I meant was "RECOVERED", no standby</pre>

<!-- Date: Tue Jun 8 10:57:59.969 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183786">183786</A>
 Terminated at 2004.06.08 10:57:35
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 10:58:34.227 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183786">183786</A>
TERMINATE: ended run after Silicon expert request
<!-- Author: Melisa & Christian (x2080) -->




<!-- Date: Tue Jun 8 11:04:48.383 2004 -->
I have marked the <b>SMXR in slot 4 of fcal00</b> online for roman pot readout.  If possible, I would like to have this board included after the pots are moved in (eg end the run while pots are moving, refresh from hardware database, and go through partition to start a new run).  If this board gives errors of any kind that impede datataking, it can be removed from the run configuration by hand (please remove just that board, not the whole crate) or can be marked offline by an expert.
<!-- Author: convery -->

<!-- Date: Tue Jun 8 11:06:25.391 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183787">183787</A>
TERMINATE: after config crates COT12, cmu00 and imu00 gave error, shepherding
didn't fix it, abort run and repartition.
<!-- Author: Melisa & Christian (x2080) -->



<!-- Date: Tue Jun 8 11:14:11.145 2004 -->
<pre>Leaving CMP HV off due to trips.  
 
Scraping for D0 now. </pre>
<!-- Author: Matt :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183788' target=_top>183788</a>) -->


<!-- Date: Tue Jun 8 11:22:20.688 2004 -->
<pre>we tried to start a new run (->CONFIG), but the crates 
b0cot12, b0imu00 and b0cmu00 gave errors, shepherding (2x) didn't 
help. We tried to repartition and then config, but again 
b0imu00 and b0cmu00 gave errors, shepherding didn't  
help again.  
The error messages are: 
 
(MLE) b0imu00:Messenger:11:16:33 AM->TDC in slot 20, ID PROM not found (blank check) 
(MLE) b0imu00:Messenger:11:16:33 AM->Resetting card in slot 20 failed 
(MLE) b0imu00:Messenger:11:16:33 AM->VISIONread/write failed in slot 20 
 
(MLE) b0cmu00:Messenger:11:14:09 AM->TDC in slot 9, ID PROM not found (blank check) 
(MLE) b0cmu00:Messenger:11:14:09 AM->Resetting card in slot 9 failed 
(MLE) b0cmu00:Messenger:11:14:09 AM->VISIONread/write failed in slot 9 
 
Paged TDC expert.</pre>
<!-- Author: Christian -->


<!-- Date: Tue Jun 8 11:30:56.903 2004 -->
<pre>CMX NE wedge 18-19 tripped while in standby.  Turned off since we were having problems anyway.</pre>
<!-- Author: Jared -->

<!-- Date: Tue Jun 8 11:33:55.144 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183788">183788</A>
TERMINATE: problems with b0cmu00 and b0imu00, no data taken.
<!-- Author: Melisa & Christian (x2080) -->
<!-- Comment: Tue Jun  8 11;35;41 comment by...Christian -->
<pre>we were stuck in config, after hitting abort
the following error popped up:

Attention !!!. FERML_SRC_FATALITY ERROR !!! 
 
 SRC Fatality Error for b0svx02: Sl 5 L2A w/o L1A 
 
 - HALT the run     
 - record any relevant information in e-log     
 - RECOVER - RUN     </pre>
<!-- Date: Tue Jun 8 11:34:54.473 2004 -->
<pre>HVAC work is done (postponed), AC is back on. 
MCR noticed Low beta quads were affecting vertical tune. 
 
TDC problems in CMU00 and IMU00, pagerd TDC experts.</pre>
<!-- Author: abw -->



<!-- Date: Tue Jun 8 11:50:32.898 2004 -->
<pre>Non-responsive TDCs in b0cmu00, slot 9 and  
b0imu00 slot 20, giving VME bus errors. 
 
Have called TDC experts. 
</pre>
<!-- Author: W.Badgett -->


<!-- Date: Tue Jun 8 11:52:53.538 2004 -->
Once again ramp up HV (except CMX and CMP), 
waiting for TDC work to start running.
<!-- Author: abw -->

<!-- Date: Tue Jun 8 11:59:34.260 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183789">183789</A>
 Activated at 2004.06.08 11:59:19
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 12:03:33.958 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183789">183789</A>
 Terminated at 2004.06.08 12:03:11
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 12:04:05.404 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183789">183789</A>
TERMINATE: still problems with IMU00, ended run.
<!-- Author: Melisa & Christian (x2080) -->



<!-- Date: Tue Jun 8 12:08:26.839 2004 -->
Could the Ace please put the plot of beam and abort gap losses 
of the last 2 hours in the elog ? Thanks alot.
<!-- Author: Lucio Cerrito -->

<!-- Date: Tue Jun 8 12:10:28.993 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183789">183789</A>
ACTIVATE: PHYSICS
<!-- Author: Melisa & Christian (x2080) -->


<!-- Date: Tue Jun 8 12:23:41.684 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183790">183790</A>
 Activated at 2004.06.08 12:23:19
<!-- Author: RunControl -->



<!-- Date: Tue Jun 8 12:28:38.415 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8542&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8542' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8543&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8543' width='300'></a></center>
<!-- Author: Matt -->


<!-- Date: Tue Jun 8 12:28:39.694 2004 -->
To contact Dan Cyr for TDC problems, use his 
cell phone 715-3704.  The TDC pager number will 
not respond.
<!-- Author: abw -->

<!-- Date: Tue Jun 8 12:30:11.216 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183790">183790</A>
ACTIVATE: PHYSICS_2_05[3,486,460]
<!-- Author: Melisa & Christian (x2080) -->



<!-- Date: Tue Jun 8 12:31:14.035 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8544&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8544' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8545&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8545' width='300'></a></center>
<!-- Author: Matt -->


<!-- Date: Tue Jun 8 12:32:13.636 2004 -->
<pre>after TDC experts fixed boards on CMU00 and IMU00, 
after activating there was a DO for COT13, shepherding 
didn't help - eun is in HALTED state, waiting for experts... 
 
Error message was: 
 crate/s: b0cot13(61),  Done TO.[RXPT] 
(MLE) b0cot13:Messenger:12:28:15 PM->TDC in slot 14, ID PROM not found (blank check) 
(MLE) b0cot13:Messenger:12:28:15 PM->Resetting card in slot 14 failed 
(MLE) b0cot13:Messenger:12:28:15 PM->VISIONread/write failed in slot 14</pre>
<!-- Author: Christian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=187390' target=_top>187390</a>) -->


<!-- Date: Tue Jun 8 12:33:53.993 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8546&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8546' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8547&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8547' width='300'></a></center>
<!-- Author: Matt -->

<!-- Date: Tue Jun 8 12:39:01.931 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183790">183790</A>
 Terminated at 2004.06.08 12:38:40
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 12:39:58.575 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183790">183790</A>
TERMINATE: still COT13 problem... abort.
<!-- Author: Melisa & Christian (x2080) -->


<!-- Date: Tue Jun 8 12:44:56.103 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183791">183791</A>
 Activated at 2004.06.08 12:44:27
<!-- Author: RunControl -->



<!-- Date: Tue Jun 8 12:48:24.346 2004 -->
<pre>We had a couple TDCs get their flash memory erased.  I was able to reload and recover the board (#381) in b0cmu00 slot 9.  The board in b0cmu00 slot #19 had to be replaced (with board #402).  The board in b0imu00 slot #20 had to be replaced (with board #228).  
 
The board in b0cmu00 slot 20 was reading DSP version 61, which is odd.  The 2 boards that were taken out of the system are now in b0diag01.</pre>
<!-- Author: Mitch -->
<!-- Comment: Tue Jun  8 12;51;22 comment by...Mitch -->
<pre>I meant to say the board in b0cmu00 slot 19 was reading DSP version 61.</pre>
<!-- Comment: Tue Jun  8 13;33;17 comment by...ronmoore -->
I was testing DSP V61 in a couple of TDC boards residing in b0diag01.  If you took the replacements from there, you might have
picked up one of the boards I was using in my tests.  And if you
tried taking a run with it, I'm sure you saw lots of reformatter errors.
<!-- Comment: Tue Jun  8 14;02;59 comment by...Mitch -->
<pre>Nope.  This board didn't come from b0diag01.  Also, this is the 3rd time in a few weeks that a board has appeared to have DSP v61.  </pre>
<!-- Comment: Tue Jun  8 16;24;25 comment by...ronmoore -->
Curious.  DSP V61 didn't exist until last week.  That's an interesting presentation of corrupted memory.
<!-- Date: Tue Jun 8 12:50:35.679 2004 -->
CMP is off; Lucio & I will have a try at turning it back on in a bit now that the HVAC is back. There were a number of trips of the cathodes in various detectors in the lower part of the hall. AT the same time, we see that the relative humidity in the hall had gone up quite a bit. I turned CMX off at this time due to this trip from standby. We will try as well to look at CMX parasitically. For the moment, both are off and should stay off.
<!-- Author: Phil Schlabach -->
<!-- Comment: Tue Jun  8 13;43;45 comment by...abw -->
CMP HV is back on, seems OK, earlier trips
probably humidity related (Hvac off period).
<!-- Comment: Tue Jun  8 13;55;38 comment by...Lucio Cerrito -->
CMP is back under the control of the shift crew. 
<!-- Date: Tue Jun 8 13:08:33.748 2004 -->
<pre>MCR calls to tell us that they will move 
Roman pots in.  (We are still down with TDC problems).</pre>
<!-- Author: abw -->


<!-- Date: Tue Jun 8 13:22:03.348 2004 -->
FYI: The slow control database do not seem to have any information  
(as accessed via runsummary web page) for CMX and CMP for last  nights' and yesterday's runs -tried 3 different runs when they were 
supposed to be on-.CSX entries do show up, though, for example.
<!-- Author: Muge -->
<!-- Comment: Tue Jun  8 13;52;45 comment by...Phil -->
I complained to Hahn about this already.
<!-- Comment: Tue Jun  8 13;56;23 comment by...Phil -->
last entry from 22:41 on 6/5
<!-- Comment: Tue Jun  8 14;04;45 comment by...Phil -->
I have a note from Lucio that we need to reboot the pc to make it come back.
<!-- Comment: Tue Jun  8 14;40;12 comment by...mku -->
great, thank you, Phil!
<!-- Comment: Tue Jun  8 16;10;23 comment by...Phil -->
This was fixed by rebooting the solenoid2 machine. The muon3pc was not rebooted.
<!-- Date: Tue Jun 8 13:28:24.072 2004 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2004&action=view&page=-8548&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2004&action=view&page=-8548  width='300' align='top'></a></td><td>Collision Hall Temperature and Humidity</td></tr></table>
<!-- Author: JJ -->

<!-- Date: Tue Jun 8 13:32:08.077 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183791">183791</A>
 Terminated at 2004.06.08 13:31:37
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 13:32:09.064 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183791">183791</A>
TERMINATE: cmp00 problem, abort run.
<!-- Author: Melisa & Christian (x2080) -->



<!-- Date: Tue Jun 8 13:32:45.302 2004 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2004&action=view&page=-8549&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2004&action=view&page=-8549  width='300' align='top'></a></td><td>Locations of temp/humidity sensors.</td></tr></table>
<!-- Author: JJ -->



<!-- Date: Tue Jun 8 13:35:23.576 2004 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2004&action=view&page=-8550&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2004&action=view&page=-8550  width='300' align='top'></a></td><td>Roman Pots were incerted again at 1:10pm in the middle of store 3562. The plot shows the positions of the pots and beam losses at B0. It appears the pots didn't affect the beam at all this time. The pots will stay there for the rest of store.</td></tr></table>
<!-- Author: K. Terashi -->

<!-- Date: Tue Jun 8 13:37:31.400 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183792">183792</A>
 Activated at 2004.06.08 13:36:50
<!-- Author: RunControl -->



<!-- Date: Tue Jun 8 13:46:33.350 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183792">183792</A>
 Terminated at 2004.06.08 13:45:40
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 13:46:34.283 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183792">183792</A>
TERMINATE: cmp problem, expert working on it. end run.
<!-- Author: Melisa & Christian (x2080) -->



<!-- Date: Tue Jun 8 13:49:16.318 2004 -->
Talked to Jim Morgan in MCR and gave him a "heads-up" that we 
have electronics problems in Collision Hall. I told him it would 
take a little while to sort it all out and decide if we would 
request to end this store early (i.e. this evening).
<!-- Author: JJ -->

<!-- Date: Tue Jun 8 13:50:44.057 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183793">183793</A>
 Activated at 2004.06.08 13:50:27
<!-- Author: RunControl -->




<!-- Date: Tue Jun 8 13:55:54.683 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183793">183793</A>
ACTIVATE: PHYSICS_2_05(3,486,460)
<!-- Author: Melisa & Christian (x2080) -->
<!-- Comment: Tue Jun  8 14;01;17 comment by...Melisa & Christian -->
<pre>the problem with the cmp00 seems solved, but we have high total deadtime ~55%.</pre>
<!-- Date: Tue Jun 8 13:58:22.115 2004 -->
<pre>We had an Event Builder error: 
 
Attention!!!. SCPU_BAD_CHANNEL_COUNTS Error !!!  
 Hardware EVB has detected a problem with data in  
 SCPU b0eb12.  
 
HRR helped.</pre>
<!-- Author: Melisa & Christian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183793' target=_top>183793</a>) -->
<!-- Comment: Tue Jun  8 14;02;19 comment by...Melisa & Christian -->
<pre>Another Event Builder error:

Attention!!!. SCPU_BAD_VRB_BYTE_COUNT Error !!! 
 Hardware EVB has detected a problem with data in 
 SCPU b0eb24. 

HRR helped.</pre>



<!-- Date: Tue Jun 8 14:03:52.579 2004 -->
<pre>We had two L2 decision TO. HRR helped.</pre>
<!-- Author: Melisa & Christian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183793' target=_top>183793</a>) -->
<!-- Comment: Tue Jun  8 14;13;03 comment by...MElisa & Christian  -->
<pre>another one, HRR fixed it.</pre>
<!-- Date: Tue Jun 8 14:05:01.722 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8551&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8551' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8552&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8552' width='300'></a></center>
<!-- Author: matt -->


<!-- Date: Tue Jun 8 14:05:34.293 2004 -->
Taking a look at CMX HV. Trigger inhibit is disabled down here. I don't think I can do any harm.
<!-- Author: Phil Schlabach -->
<!-- Comment: Tue Jun  8 14;10;49 comment by...ps -->
West 22,23 cathode on. Anode on Droege. The rest is off.
<!-- Comment: Tue Jun  8 14;20;55 comment by...ps -->
Cathode is holding.
<!-- Comment: Tue Jun  8 14;36;21 comment by...ps -->
ok after 10 min. on the Droege. Now connected to the CAEN and at full.
<!-- Comment: Tue Jun  8 14;37;16 comment by...ps -->
Turning on 0,1 W for comparison
<!-- Comment: Tue Jun  8 14;45;07 comment by...ps, iz -->
now also turning on 13, 14 W
<!-- Comment: Tue Jun  8 14;54;42 comment by...ps -->
turning on everything
<!-- Date: Tue Jun 8 14:07:25.803 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8553&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8553' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8554&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8554' width='300'></a></center>
<!-- Author: matt -->



<!-- Date: Tue Jun 8 14:12:48.282 2004 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2004&action=view&page=-8555&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2004&action=view&page=-8555  width='300' align='top'></a></td><td>We have a hot trigger tower in the Plug, eta=0, phi=15, new  
since the power glitch.  This is causing an enormous increase  
in the L2 rate, JET_40 trigger.  Carla, please help us! 
<P> 
Dead time is about 60 percent. 
</td></tr></table>
<!-- Author: W.Badgett -->
<!-- Comment: Tue Jun  8 14;13;22 comment by...WB -->
The hot tower is not seen in the plug (ADMEM) readout.

<!-- Comment: Tue Jun  8 15;01;24 comment by...Vadim R -->
Jet40 trigger rate at L2 was the one causing the problem. We did not see any high occupancy for L1 Dirac plot. Also, I couldn't see any L1 hot trigger so I don't think this was a L1 related issue. Starting a new run solved everything, so maybe the problem was in the initialization of the DCAS board. Looking at the L1 Trigmon error logs it seems that the DCAS readout always had the MSB for the had energy for this tower set. 




<!-- Date: Tue Jun 8 14:19:38.553 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183793">183793</A>
 Terminated at 2004.06.08 14:18:41
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 14:19:40.044 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183793">183793</A>
TERMINATE: had a problem with high deatime, end of run as requested by L1 expert.
<!-- Author: Melisa & Christian (x2080) -->



<!-- Date: Tue Jun 8 14:29:20.048 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183795">183795</A>
TERMINATE: no data
<!-- Author: Melisa & Christian (x2080) -->


<!-- Date: Tue Jun 8 14:32:04.265 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
 Activated at 2004.06.08 14:31:51
<!-- Author: RunControl -->



<!-- Date: Tue Jun 8 14:32:05.210 2004 -->
<pre>Marked TDC in slot 6 of b0cot12 as offline, changing  
TdcReadoutMode to 2.   Should be switched back when  
recovered.</pre>
<!-- Author: W.Badgett :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796' target=_top>183796</a>) -->

<!-- Date: Tue Jun 8 14:34:16.993 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
ACTIVATE: PHYSICS_2_05[3,486,460]
<!-- Author: Melisa & Christian (x2080) -->






<!-- Date: Tue Jun 8 14:42:18.937 2004 -->
<pre>BMU trip, SW 156 
</pre>
<!-- Author: MAtt :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796' target=_top>183796</a>) -->


<!-- Date: Tue Jun 8 14:42:32.940 2004 -->
<pre>triggr inhibit from HV trip in BMU, put run to halt 
and back to active after trip was recovered.</pre>
<!-- Author: Chrsitian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796' target=_top>183796</a>) -->





<!-- Date: Tue Jun 8 15:06:21.115 2004 -->
<pre>we had several done TO in CMX00, we tried to 
shepherd it several times but didn't work. Put run to 
halt while TDC expert will try to fix the problem.</pre>
<!-- Author: Melisa :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796' target=_top>183796</a>) -->

<!-- Date: Tue Jun 8 15:08:57.329 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
 Terminated at 2004.06.08 15:08:44
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 15:09:44.393 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
TERMINATE: CMX00 TDC problem, ended run.
<!-- Author: Melisa & Christian (x2080) -->


<!-- Date: Tue Jun 8 15:16:19.146 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183799">183799</A>
 Activated at 2004.06.08 15:16:12
<!-- Author: RunControl -->


<!-- Date: Tue Jun 8 15:18:08.004 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183799">183799</A>
ACTIVATE: PHYSICS_2_05[3,486,460]
<!-- Author: Melisa & Christian (x2080) -->



<!-- Date: Tue Jun 8 15:18:53.211 2004 -->
<p>Been working all day (and thruogh the delicious MCS meeting!) on a handful of 
ICICLE problems:</p> 
 
<ul> 
<li>Over the weekend, Rob Harr found that all CLC HVs stored in the run condition 
DB are zero.  Talked with Mary Convery and Rob about this.  ICICLE had no  
complaints in its log window.  Opened the local iFIX 
database on CLC1 and found...the values are all zero.  Mary will have to fix how  
these tags are filled; this must have been broken in the move to Windows  
2000.</li> 
<li>With the muon HV problems today due to humidity, we found that all MUON3 
tags were not being written by ICICLE--as seen both in the run condition DB and 
in Larry Kirschs ICICLE plotting page.  Phil looked back and found this was the case 
since Saturday.  I looked on the iFIX Network Status picture, and found there was 
no communication between MUON3 and SOLENOID2 (where ICICLE runs), but  
MUON3 was seen by other nodes.  Again, the ICICLE log showed no problem. 
We rebooted SOLENOID2, and communications appears to be restored.  We assume 
tags will be read by ICICLE now, but we will check there are no problems.</li> 
<li>At least in the second case above, we should have gotten some warning.   
Margherita will change ICICLE so it complains if it reads no tags from any node 
during the hourly full readout.</li> 
</ul>
 
<p>These problems point to a larger problem: the detector groups are not  
checking that the proper info is being written into ICICLE.  I will work on this.</p>
<!-- Author: Steve Hahn -->


<!-- Date: Tue Jun 8 15:20:49.488 2004 -->
<pre>we had a consumer error: 
 
 Attention !!!. CER_SVXMON_HALT_RECOVER_RUN_ERROR !!!  
  
 Stuck Cellid I/B0/W0/L0/C4-7 .  
 AUTO HRR will be issued   
 
HRR fixed it.</pre>
<!-- Author: Melisa & Christian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183799' target=_top>183799</a>) -->


<!-- Date: Tue Jun 8 15:21:20.028 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8556&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8556' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-8557&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-8557' width='300'></a></center>
<!-- Author: matt -->

<!-- Date: Tue Jun 8 15:23:50.818 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
central: <tt>COT SL 1, 5, and 8 each have a new group of empty cells, due to <br>
the exclusion of b0cot12 (COT_12) slot 6, and b0cot13 (COT_13) slot 14 <br>
from the readout.</tt>
<!-- Author: Consumer Operator( William Badgett) -->


<!-- Date: Tue Jun 8 15:23:52.070 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
cal: <tt>Hot towers in "CEM Average Raw ADC per hit" histogram, eta=-9 (West) <br>
and phi=1; and eta=0 (East), phi=22.<br>
Hot tower in "CHA Average Raw ADC Counter Per hit" histogram at eta=-6<br>
(West) and phi=1.</tt>
<!-- Author: Consumer Operator( William Badgett) -->


<!-- Date: Tue Jun 8 15:23:53.982 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
trigger: <tt>XTRPMon Summary looks messed up, ala TrigMon problem seen <br>
previously.<br>
L1_MET25 cross section is high; L2_MET35 and L2_EM40 and L2_EM70 and <br>
L2_EM90 are also high; perhaps related to hot calorimeter towers?</tt>
<!-- Author: Consumer Operator( William Badgett) -->

<!-- Date: Tue Jun 8 15:29:40.235 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796">183796</A>
monitors: <tt>Surprisingly, Stage0 passed for run 183796, in spite of the new <br>
holes in the readout.</tt>
<!-- Author: Consumer Operator( William Badgett) -->



<!-- Date: Tue Jun 8 15:30:34.115 2004 -->
<pre>Summary of TDC Problems: 
 
Sometime after run was stopped this morning  
(~9am), something happened to cause a number  
of TDCs in the system to lose their FLASH RAM 
programming.  This event could be associated  
with power glitch that occured shortly after  
that time, but there is no way to verify this  
possibility.  A similar event happened about  
two weeks ago (on May 22nd) in association  
with a power outage that occured during a  
thunderstorm.  In the earlier case, we found  
five ECL muon boards with corrupted FLASH  
memories.  4/5 were recoverable simply by  
re-programming the FLASH RAM and the fifth  
needed to be replaced. 
 
Today the problem was more widspread.  We 
also obsevered corrupted COT LVDS boards  
in the collision hall.  These three boards  
(cot13/slot6,cot13/slot14, and cot12/slot6) 
are not recoverable from upstairs and will  
need to be replaced on access.  We also had  
a significant number of corrupted boards  
upstairs.  A total of four board upstairs  
have been replaced (cmp00/slot5,cmu00/slot19, 
cmx00/slot13, and imu00/slot20).  some of  
these were completely non-recoverable and 
other were reloadable but still had errors 
in DAQ mode.  For example, the board in  
cmp00/slot5 caused L3 errors presumably due 
to incorrect formatting of the data by the  
DSP.  The board in cmx00/slot13 worked for  
a while but then gave header mis-match  
errors. 
 
We also saw a number of problems in other  
muon/hadron tdc crates when we first tried  
to start running again.  To be safe, we  
re-programmed the FLASH memories on all  
the TDC boards in crates on the 1st floor 
(except for b0musc00 which was an oversight). 
We also reporgrammed the FLASH memories of 
all the TDC boards in cot12 and cot13 since 
those crates also seemed to be affected.  
 
We will attempt to diagnose problems with  
pulled boards later this evening before  
the upcoming access.         </pre>
<!-- Author: Eric James & Mitch Soderberg -->


<!-- Date: Tue Jun 8 15:43:12.417 2004 -->
CMX status:  the problem channels of this morning (21,22W & 12,13 W) seem to be fine. There are a handful of other channels in the lower part of the detector on each side which aren't holding HV. Our belief is that they are still drying out as that's how they act. We've left these handful either at standby or off. We'll try again in a while. <b>Meantime, CMX HV control is expert only.</b>
<!-- Author: Phil, Ingyin, Dale -->

<!-- Date: Tue Jun 8 15:55:31.024 2004 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183786>
183786</A> 
x2CDEA</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_2_05 [3,486,460]</TD>                                       
<TD ALIGN=CENTER>04:38:27</TD>                                          
<TD ALIGN=CENTER>10:57:35</TD>                                          
<TD ALIGN=CENTER>04:15:09</TD>                                          
<TD ALIGN=RIGHT>    226,978,600</TD>                                    
<TD ALIGN=RIGHT>     3,009,201</TD>                                     
<TD ALIGN=RIGHT>     794,923</TD>                                       
<TD ALIGN=RIGHT>  408.868</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183793>
183793</A> 
x2CDF1</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_2_05 [3,486,460]</TD>                                       
<TD ALIGN=CENTER>13:50:27</TD>                                          
<TD ALIGN=CENTER>14:18:41</TD>                                          
<TD ALIGN=CENTER>00:12:23</TD>                                          
<TD ALIGN=RIGHT>      3,061,618</TD>                                    
<TD ALIGN=RIGHT>       699,680</TD>                                     
<TD ALIGN=RIGHT>      28,915</TD>                                       
<TD ALIGN=RIGHT>   13.212</TD>                                          
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183796>
183796</A> 
x2CDF4</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_2_05 [3,486,460]</TD>                                       
<TD ALIGN=CENTER>14:31:51</TD>                                          
<TD ALIGN=CENTER>15:08:44</TD>                                          
<TD ALIGN=CENTER>00:13:36</TD>                                          
<TD ALIGN=RIGHT>      4,429,275</TD>                                    
<TD ALIGN=RIGHT>        72,019</TD>                                     
<TD ALIGN=RIGHT>      24,719</TD>                                       
<TD ALIGN=RIGHT>   14.485</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=183799>
183799</A> 
x2CDF7</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_2_05 [3,486,460]</TD>                                       
<TD ALIGN=CENTER>15:16:12</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>00:34:31</TD>                                          
<TD ALIGN=RIGHT>     19,438,921</TD>                                    
<TD ALIGN=RIGHT>       251,906</TD>                                     
<TD ALIGN=RIGHT>      79,134</TD>                                       
<TD ALIGN=RIGHT>   36.843</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>15:55:06</TD>                                                       
<TD ALIGN=CENTER>05:15:40</TD>                                          
<TD ALIGN=RIGHT>     253,908,414</TD>                                   
<TD ALIGN=RIGHT>      4,032,806</TD>                                    
<TD ALIGN=RIGHT>       927,691</TD>                                     
<TD ALIGN=RIGHT>  473.408</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Tue Jun 8 15:56:41.206 2004 -->
<font color=red>Shift Summary: </font><pre> Another productive day shift: 
 
  Started with store 3562, L=25 10**30, R183786, CMX off. 
  Glitch in ComEd (9:19) tripped 345KV supply, caused LRF1 HV to trip; 
  huge glitch in losses, RFSUM and TEL trip off.  After 
  glitch, TEL tripped 3 times, PAGC went red, and tuning 
  with collimators continued from 920 through 1150. 
 
  HVAC in B0 hall was turned off at 9:54 for fan replacement, 
  which was in fact not completed. However, humidity and 
  temperature in B0 went up, resulting in CMP trips 
  (with CMP on stby), and also motion of low beta quads, 
  which caused losses for D0.  HVAC was restored- this eventually fixed 
  CMP trips and D0 losses, and resolved confusing situation for 
  MCR and Muon experts. 
 
  MCR moved CDF Roman pots into position.  This had no 
  adverse effects on losses.  L2 roman pot+jet trigger 
  has counts. 
 
  We could not turn on because of loss of 3 TDC boards 
  Cot12, CMU00, IMU00, flash memory erased.  Additional 
  COT boards in B0 will not respond, need access at end of 
  store.  Cot_12, Cot_13 have occupancy holes as a result. 
 
  While testing TDC repairs, R183793, we got high L2 deadtime 
  caused by hot PHA channel (0,15) which showed up in dcas/admem 
  comparison.  This was fixed by experts Vadim+Vivek, 
  by starting new run (stuck bit read out by L2 gave 
  a 20 microbarn L2 accept rate, though L1 rates were OK). 
  As of 183799, we have Muon HV on, including CMX, 
  with TDC problems apparently fixed except 
  for those in collision hall. 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   183786,793,796,799
Delivered Luminosity   579  
Acquired Luminosity    178  
Efficiency             31%
   
</td></tr></table></pre>
<!-- Author: abw -->