<!-- Title: Day shift. Sat Jun 16, 2001 -->
<!-- SciCo: Tom LeCompte -->
<!-- DAQAce: Alexi Sedov -->
<!-- MonAce: Andras Zsenei -->
<!-- CO: Masa Tanaka -->
<!-- OpManager: Doug Benjamin -->
<!-- Notes: -->
Studies followed by Shot Setup at 1200.  In the interim, have Rick look at endwall power supplies, Ivan at the TOF, Peter W. at the L2 decision crate, L3 to do whatever it is they have to do, CLC QiE calibration...reintegrate by 1100.


<!-- Date: Sat Jun 16 08:42:35 2001 -->
We page L3 to let them know that there will be time for them to work on the system.
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 08:54:24 2001 -->
IFIX PC's are acting up, so before rebooting, we page JC to talk us through the procedure.  We don't want to leave the system in a funny state (like happened the last time I did this!)
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 08:56:57 2001 -->
Page Level 3 again.  
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 09:06:08 2001 -->
<pre>Worried by the absence of the audio alarms from iFIX 
recorded by the previous shift crew we decided to logout and 
log back on to see if the situation gets better. We got some 
typical Windows behaviour. After hitting Log-Off the screen  
became shaded and the whole system hung (no response to  
CTRL-ALT-DEL). We paged Y.C.Yun and he instructed us to 
reboot the computer (we did a hardware reset ;) ). After 
the booting the system looked ok.</pre>
<!-- Author: Andras -->


<!-- Date: Sat Jun 16 09:26:08 2001 -->
TeV quench on abort.  No estimate.  Set up for quiet time calibrations.  WCAL04 is still off - we'll just have to go  
without that.
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 10:10:40 2001 -->
<pre>Calibrations : COT 117870, SMX 117876, Muon 117878. 
</pre>
<!-- Author: Alex Sedov :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=117870' target=_top>117870</a>) -->


<!-- Date: Sat Jun 16 10:29:10 2001 -->
<pre>I tried to do TOF calibrations 4 times (run ## 117872-5) w/o any sucsess to find those runs in the database.</pre>
<!-- Author: Alex Sedov :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=117872' target=_top>117872</a>) -->


<!-- Date: Sat Jun 16 10:46:36 2001 -->
<pre>Cal QIE calibration 117871. 
From error file :  
 
(b0ccal09) QIE calib: slot 21, qie 01:Fail 14: Sigma(off) big     , val 01.0946, cap 1, rng3.</pre>
<!-- Author: Alex Sedov :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=117871' target=_top>117871</a>) -->


<!-- Date: Sat Jun 16 10:48:14 2001 -->
<pre>Calibrations : LED 117880, Xenon 117882/117883.</pre>
<!-- Author: Alex Sedov :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=117880' target=_top>117880</a>) -->


<!-- Date: Sat Jun 16 10:52:39 2001 -->
<br><b>b0wcal04 Power:</b> 
<br> Reset power for this crate one more time at 09:55, but  
want to keep a close eye on the crate.  Requested that the  
shift crew notify R. Tesarek just prior to injection. 
<br>
<!-- Author: R.J. Tesarek -->


<!-- Date: Sat Jun 16 10:59:54 2001 -->
Ming says CLC has some bad Admems, and they won't be able to have their system working again in under an hour.  We inform MCR of the status (they use our luminosity during shot setup).
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 11:23:20 2001 -->
<pre>PSM alarm: 2nd floor 2RR30C has channel 0 on O V instead of +5,  
and channel 1 has 0.85 V on it instead of 12 V; 2RR22I_1 has 
+10, +5.18 and -7.46 instead of +5, +12 and -5.2; 2RR22I_2 
has all its voltages on 0 V. After inspecting the crates it 
turned out that experts were working on 2RR30C and power  
cycling it... The other crate looked fine (it is the one with 
the scope to monitor the protons and antiprotons). Paged 
Fred Lewis</pre>
<!-- Author: Andras -->


<!-- Date: Sat Jun 16 11:24:37 2001 -->
<pre>Suddenly 4 BSC alarms in 4 minutes. All of them recovered in 
a few seconds.</pre>
<!-- Author: Andras -->
<!-- Comment: Sat Jun 16 11;27;16 comment by...Andras -->
<pre>2 more alarms. Paged the expert.</pre>


<!-- Comment: Sat Jun 16 11;29;47 comment by...Andras -->
<pre>Expert came in and said that he was adjusting the voltages, so
this behaviour was normal.</pre>
<!-- Date: Sat Jun 16 11:27:48 2001 -->
<pre>Here's the agreement between me, Nigel and the L1/L2 group.  They  
can have L1 and L2 until the first real bunch goes in, after  
which they will stop immediately and we will reintegrate the  
system and prep for the store.   
 
This is a risk (I estimate a 50-50 chance we'll be ready when we 
have the first collisions) but it was the best compromise we  
could broker between the desire to get Level 2 working and the 
desire to provide data to the rest of the collaboration. 
 
 
</pre>
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 11:28:16 2001 -->
<pre>CLC alarm. Cleared itself in a few seconds (before the detailed 
window came up so could not determine which part has tripped)</pre>
<!-- Author: Andras -->



<!-- Date: Sat Jun 16 11:56:53 2001 -->
<pre>Fix CLC channels 
---------------- 
 
From the data taken last night we observed problem with a few 
channels (ch 19,26,35). Initially we thought that it could be 
que to bad qie constants loaded in ADMEM. This morning after 
Tevatron dumped the beams we performed some test and found that 
the channels were broken in the transition board. We swapped 
the bad boards with good ones, re-calibrate these channels 
(qie calibration), and checked with LED. Now all channels look 
okay. </pre>
<!-- Author: Saha and Ming -->


<!-- Date: Sat Jun 16 12:11:54 2001 -->
<pre>I studied the CMX noise problem on the east side.  It is clearly correlated with a particular CAEN channel used by wedges 18&19 aka the 30deg miniskirt section.  I swapped this with the channel for wedge 20 which currently is unused (aka the 10deg miniskirt section).  The scope shows no differential noise on the east side.   The noise on the west side has some other source.</pre>
<!-- Author: Michael Schmitt -->


<!-- Date: Sat Jun 16 12:14:28 2001 -->
<pre>Level 2 work is done.  We try and set up for a Jimmy run,  
hoping to beat the injection of beam in the Tevatron.</pre>
<!-- Author: TJL -->
<!-- Comment: Sat Jun 16 15;35;00 comment by...Peter Wittich -->
<a href=
'/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=10&frame=2&anchor=15:15:23'>
L2 log entry describing today's work</a> 
<!-- Date: Sat Jun 16 12:28:55 2001 -->
<pre>We tried the new jpython script for the TOF TAC calibration.  
we have been able to do only the first iteration, more 
investigation is needed to see what is going on. After  
that we did successfully two TAC calibrations, the 
first one with all the TOAD channels enabled and the second 
one just enabling the channels 0,3 and 5. From this last 
calibration we saw that there is not time-time cross talk 
at all. 
 </pre>
<!-- Author: Ivan  -->


<!-- Date: Sat Jun 16 12:36:20 2001 -->
<pre>Turning on CES/CPR/CCR for Jimmy runs.</pre>
<!-- Author: Andras -->
<!-- Comment: Sat Jun 16 12;51;47 comment by...Andras -->
<pre>Beam is expected now any minute... ending Jimmy run, so
I'm putting CES/CPR/CCR to standby.</pre>
<!-- Date: Sat Jun 16 12:49:14 2001 -->
<br><b>BSC ADMEM calibration:</b> 
<br>While attempting to perform BSC ADMEM QIE calibration (standalone program) we learn that the ADMEM in slot 16 has 
3 dead CAFE cards.  These cards digitize the signals for BSC 
counters BSC1-2, BSC1-3, BSC1-4, all on the east side.   
<br><br> 
We will attempt to replace these channels prior to shot setup. 
<br>
<!-- Author: R.J. Tesarek, K. Terashi -->



<!-- Date: Sat Jun 16 12:54:50 2001 -->
<pre>Jimmy runs : 117905 10K events (standby), 117906 12K events (on).</pre>
<!-- Author: Alex Sedov :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=117905' target=_top>117905</a>) -->


<!-- Date: Sat Jun 16 13:23:04 2001 -->
MCR calls.  Beam is about to go in the Tevatron.
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 13:23:12 2001 -->
<br><b>BSC ADMEM update:</b> 
<br> Problems observed earlier turn out to be upstream of the 
ADMEM.  The BSC transition board will likely have to be replaced. 
The data from BSC1-E channels 2-4 will be essentially useless. 
<br>
<!-- Author: R.J. Tesarek K. Terashi -->


<!-- Date: Sat Jun 16 13:30:30 2001 -->
<pre>Taking the magnet down to pump the Watt can. 
We are at 5.7 um, we need to pump at 7, and we will 
trip at 7.5.  It's growing by 3 a day, so it's clear  
that if we don't pump now, we won't make it through 
the store. </pre>
<!-- Author: TJL -->
<!-- Comment: Sat Jun 16 13;41;30 comment by...TJL -->
We were puzzled about why this didn't alarm, but we think we understand this - the magnet alarms at full field (the "sonar pings").  When silenced, the magnet doesn't un-silence itself during a dump.
<!-- Date: Sat Jun 16 13:31:46 2001 -->
<pre>The whole COT has tripped, and TOF as well (even though it 
seems to me that TOF was not powered!!!!!). COT was on standby.</pre>
<!-- Author: Andras -->
<!-- Comment: Sat Jun 16 13;34;02 comment by...Andras -->
<pre>Actually we are having problems with the magnet, so these 
alarms make sense (paged COT in order to inform them about 
the situation)</pre>
<!-- Comment: Sat Jun 16 14;14;38 comment by...Andras -->
<pre>TOF expert explained that whenever the magnet is not at full
power there is a TOF alarm even if TOF is not powered!</pre>
<!-- Date: Sat Jun 16 13:33:32 2001 -->
<pre>Message from Error display : 
 
(MLE) b0l2de00:Messenger:1:11:46 PM-&gt;Alpha in slot 3 is no longer processing at spot 5  
 
Message from alpha_check : 
 
Info: Alpha is okay. Currently at spot 100: Waiting for L1A 
 TSI handshake bits are 4fff0000 
 TSI startload bits are 10101c6 
 TSI MODDONE bits are 4197f42f 
 Number of words for TL2D, buffer 0: 25010000 
 Number of words for TL2D, buffer 1: 25010000 
 Number of words for TL2D, buffer 2: 25010000 
 Number of words for TL2D, buffer 3: 25010000 
 Address for DMA buffer 0 is  500080 
 Address for DMA buffer 1 is  500100 
 Address for DMA buffer 2 is  510600 
 Address for DMA buffer 3 is  510f80 
 Address for DMA buffer 4 is  514580 
 Address for DMA buffer 5 is  518d80 
 Address for DMA buffer 6 is  100000 
 Address for DMA buffer 7 is  100000 
 Address for DMA buffer 8 is  100000 
 Address for DMA buffer 9 is  100000 
 Executable name is  l2exe/36x36_1_Coupled_noInclusive_v-1_B0L2PR00 
 
So is alpha OK? And why did RC send that message?</pre>
<!-- Author: Alex Sedov :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=117907' target=_top>117907</a>) -->



<!-- Date: Sat Jun 16 13:39:29 2001 -->
<pre>Java OutOfMemoryError. HAve to restart RC.</pre>
<!-- Author: Alex Sedov :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=117907' target=_top>117907</a>) -->



<!-- Date: Sat Jun 16 13:50:09 2001 -->
<pre>We heard a "Node Unreachable" alarm but none of the nodes 
was/turned purple!! </pre>
<!-- Author: Andras -->


<!-- Date: Sat Jun 16 14:12:20 2001 -->
<pre>iFIX: 
Those 'heartbeat' and 'node unreachable' are checked every 1 minutes to initiate sound alarms. Sometimes these alarms are 
present only a short period of time ( &lt;1min) and you may not hear sound alarms. Alarms of this nature are not crucial so 
please do not worry even if there is no sound alarm. </pre>
<!-- Author: jc yun -->



<!-- Date: Sat Jun 16 14:21:08 2001 -->
Watt can pumped.  Magnet goping back up.
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 14:36:31 2001 -->
<pre>PSM alarm for East EndWall CONET_4. </pre>
<!-- Author: Andras -->
<!-- Comment: Sat Jun 16 14;47;22 comment by...TJL -->
(Same endwall PS as last night)
<!-- Date: Sat Jun 16 14:47:20 2001 -->
<br><b>b0wcal04, power supply fault:</b> 
<br>The +5V analog power again faulted.  We observe that the 
fault occurred when beam was in the machine.  Reset the supply 
and watch what happens when beam returns 
<br>
<!-- Author: R.J. Tesarek -->



<!-- Date: Sat Jun 16 15:13:51 2001 -->
<pre>CONET_4 has tripped again. Rick has been notified.</pre>
<!-- Author: Andras -->


<!-- Date: Sat Jun 16 15:14:23 2001 -->
<pre>CEM/CHA/WHA heartbeat alarm for 10-15 seconds.</pre>
<!-- Author: Andras -->


<!-- Date: Sat Jun 16 15:26:35 2001 -->
<br><b>b0wcal04, power supply fault:</b> 
<br> Reset power supply again.  The previous trip corresponded 
roughly to beam being injected.
<!-- Author: R.J. Tesarek -->


<!-- Date: Sat Jun 16 15:34:24 2001 -->
<pre>Stopped run 117911. Had 13 filters dead and 1 in old state.</pre>
<!-- Author: Alex Sedov -->



<!-- Date: Sat Jun 16 15:35:14 2001 -->
Proton losses look terrible.  They injected 1700 mA half an hour ago and less than 980 mA remain.
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 15:49:15 2001 -->
Magnet up.  Call MCR to notify them.  They also inform us the tunes are crossed; they'll dump the protons and shoot again.  
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 15:49:42 2001 -->
<pre>CEM/CHA/WHA heartbeat alarm. Goes away within a minute.</pre>
<!-- Author: Andras -->


<!-- Date: Sat Jun 16 15:54:12 2001 -->
Protons dumped.
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 15:55:27 2001 -->
<font color=red>Shift Summary: </font><pre>Pumped the Watt can, many minor repairs,  
mostly waiting for beam.  WCAL04 still has 
power supply problems.</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   
Delivered Luminosity   0  pb-1
Acquired Luminosity    0  pb-1
Efficiency             100
   
</td></tr></table></pre>
<!-- Author: TJL -->


<!-- Date: Sat Jun 16 16:02:18 2001 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2001&action=view&page=-1142&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2001&action=view&page=-1143  width='300' align='top'></td><td>The SVRAD plot for the shift. The monitors have been reset just before 10 am, but the net gain from 8 am is compatible with 0. The numbers from the end of the shift are the  following                (Rads) W Inner BLM    .486003 W Outer BLM    .0186144 E Inner BLM    1.324441 E Outer BLM    1.748846</a></td></tr></table><font color=red><cite>(entry outside this shift's time range )</cite></font>
<!-- Author: Andras -->
<!-- Comment: Mon Jun 18 09;51;14 comment by...R. Vidal -->
the thumbnail for this image was not created automatically because it was captured as a jpg file when it was actually a gif image;  I made the thumbnail by hand.
<!-- Date: Sat Jun 16 16:06:02 2001 -->
<pre>The CO found that LumMon was crashing frequently. CLC expert 
was contacted, and found that this was due to corrupt data 
in the CLLD bank. The process 
<!-- Author: Ming -->


<!-- Date: Sat Jun 16 16:06:49 2001 -->
<pre>The CO found that LumMon was crashing frequently. CLC expert 
was contacted, and found that this was due to corrupt data 
in the CLLD bank. The process 
<!-- Author: Ming -->
<!-- Comment: Sat Jun 16 16;12;02 comment by...Ming -->
<pre>(continue from what was left out) ... The process "receive" that
is supposed to be running in CLC00 cpu, which fills CLLD bank
with proper data, had stopped. We reset the crate to start up the
"receive" process. </pre>