<!-- Title: Day shift. Mon Apr 21, 2003 -->
<!-- SciCo: Mako Shimojima -->
<!-- DAQAce: Steve Vejcik -->
<!-- MonAce: Thorsten Walter -->
<!-- CO: Davit Chokheli -->
<!-- OpManager: Masa Tanaka -->
<!-- Notes: -->
<pre>Store 2445 w/ lum 14.3E30 @ B0 
Pbar stack at 134 mA + 6.5 mA/hour 
</pre>


<!-- Date: Mon Apr 21 08:35:18.401 2003 -->
SVX AVDD B1 W2 L2 went pink, while run stuck. 
Unmarked it.  
<!-- Author: Thorsten :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161793' target=_top>161793</a>) -->

<!-- Date: Mon Apr 21 08:38:48.881 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161793">161793</A>
 Terminated at 2003.04.21 08:37:39
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 08:38:58.856 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161793">161793</A>
TERMINATE: failed to go to active on new run. Busy timeouts-checking log files while trying
again,.
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 08:40:52.454 2003 -->
<pre>Following message as run ending: 
1 crate/s: b0svx02(112),  busy.38 crate/s: b0imu00(80), b0imu01(81), b0cmu01(82), b0cmu00(83), b0cmp00(84), b0musc00(85), b0cmx00(86), b0cmx01(87), b0xtrp00(96), b0xft01(98), b0xft03(100), b0xft05(102), b0clc00(118), b0svt00(120), b0svt01(121), b0svt07(122), b0svt02(123), b0svt06(124), b0svt03(125), b0svt05(126), b0svt04(127), b0l1cl00(128), b0l1cl01(129), b0l1cl02(130), b0l1cl03(131), b0l1cl04(132), b0l1cl05(133), b0l1gl00(134), b0tsi03(138), b0puls00(139), b0l2cl00(144), b0l2cl01(145), b0l2cl02(146), b0l2cl03(147), b0l2cl04(148), b0l2cl05(149), b0l2de00(150), b0htdc00(153),  in error.[RXPT]</pre>
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161793' target=_top>161793</a>) -->

<!-- Date: Mon Apr 21 08:44:33.281 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161794">161794</A>
 Activated at 2003.04.21 08:42:51
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 08:44:34.650 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161794">161794</A>
ACTIVATE: trying to restart a new physics run
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 08:49:20.933 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161794">161794</A>
 Terminated at 2003.04.21 08:48:20
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 08:49:33.586 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161794">161794</A>
TERMINATE: Need to re-start evb. unrecoverable run start.
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 08:54:47.330 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3786&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3786' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3787&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3787' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3788&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3788' width='300'></a></center>
<!-- Author: Thorsten -->


<!-- Date: Mon Apr 21 08:57:26.462 2003 -->
<pre>L3/EVB (Ilya) experts here. Reset run, cleaned  up  EVB and 
try to start run again.</pre>
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795' target=_top>161795</a>) -->

<!-- Date: Mon Apr 21 08:58:30.303 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795">161795</A>
 Activated at 2003.04.21 08:57:29
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 08:58:40.131 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795">161795</A>
ACTIVATE: Try a physics run with cleaned up EVB
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 09:04:19.147 2003 -->
<pre>Changing prescales by hand to boost rate into L3 for 
tests of L3 errors with Ilya.</pre>
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795' target=_top>161795</a>) -->


<!-- Date: Mon Apr 21 09:16:35.755 2003 -->
SVX and L00 alarm in iFix window, probably bus error in crate cr_n 5. Alarm cleared after a few seconds.
<!-- Author: Thorsten :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795' target=_top>161795</a>) -->
<!-- Comment: Mon Apr 21 10;14;24 comment by...Thorsten -->
This reoccured; alarms cleared immediately
<!-- Date: Mon Apr 21 09:27:20.633 2003 -->
The Myron Mode Plug laser calibration data is not being written to disk. I am investigating and would like to do plug laser calibration runs as soon as possible. 
Please call me at X2144 (trailers) or at X4228 (home) and e-mail me 
 at pauletta@fnal.gov as soon as it becomes clear when I might have some time to do it.  
<!-- Author: Giovanni Pauletta -->

<!-- Date: Mon Apr 21 09:27:45.678 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795">161795</A>
 Terminated at 2003.04.21 09:26:39
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 09:27:52.423 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795">161795</A>
TERMINATE: move on another test
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 09:41:22.949 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161796">161796</A>
 Activated at 2003.04.21 09:39:57
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 09:41:23.185 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161796">161796</A>
ACTIVATE: No Silicon run testing L3 with adjusted prescales
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 09:47:56.993 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161796">161796</A>
 Terminated at 2003.04.21 09:47:02
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 09:48:44.696 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161796">161796</A>
TERMINATE: L3 going into error without Si but with high rate.
<!-- Author: Steve2080 -->
<!-- Comment: Mon Apr 21 09;58;39 comment by...Gino -->
<table border=0><td><tr><td bgcolor=#e0e030>OF COURSE!</td></tr></td></table>
<!-- Date: Mon Apr 21 09:48:56.824 2003 -->
<pre>Plan of the day and beyond: 
 
- Stack till 150 mA and shoot. 
- Starting from tomorrow 08:00, 
  5 shifts of accelerator studies. 
 
Current pbar stack: 142 mA + 6.4 mA/hour. 
 
</pre>
<!-- Author: mako -->
<!-- Comment: Mon Apr 21 10;01;52 comment by...mako -->
<pre>MCR called: One hour to go...</pre>
<!-- Comment: Mon Apr 21 10;25;23 comment by...mako -->
<pre>No Quiet Time. They are going to
shot setup right away...</pre>
<!-- Date: Mon Apr 21 09:55:38.865 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3789&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3789' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3790&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3790' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3791&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3791' width='300'></a></center>
<!-- Author: Thorsten -->



<!-- Date: Mon Apr 21 10:04:22.172 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797">161797</A>
 Activated at 2003.04.21 10:03:28
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 10:04:46.358 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797">161797</A>
ACTIVATE: Start physics run with no silicon again
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 10:05:29.904 2003 -->
This is another run with hand-adjusted prescales
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797' target=_top>161797</a>) -->



<!-- Date: Mon Apr 21 10:15:39.577 2003 -->
<pre>Bunch Counter Error Done Timeout b0cot17: 
b0cot17:Messenger:10:13:40 AM->Event 101390: Bunchcounters in slot 3 (BC=18) and slot 16 (BC=65) disagree (at least)</pre>
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797' target=_top>161797</a>) -->



<!-- Date: Mon Apr 21 10:40:09.973 2003 -->
<pre>Error Handler (EH) has been changed. The new version v1_09_5 is now current. 
When possible, please close EH GUI and restart a new one from RC Partitioning Menu to pick up this last version. Remember to bring EH in sync with the rset of the DAQ with 'Special Needs Client' option in the Local Client Controller Gui. 
This version's highligths: 
- a counter and time stamp has been added to the orange pop-up windows  
- a new YELLOW warning pop-up window has been added for FERML_TDC_NOT_READY_GLOBAL (ie. Timeout on Global TDC Done) 
with instructions on what to do (most likely to shepherd the crate if regular HRRs did not work)  
- the color of the SCPU_CANT_RESET_VRB error pop-up window has been changed from orange to YELLOW as well. 
 
Please report any problems. Thanks, 
 
</pre>
<!-- Author: Donatella T. -->
<!-- Comment: Mon Apr 21 11;08;12 comment by...steve -->
Restart Error handler on partition 0.
<!-- Date: Mon Apr 21 10:43:38.665 2003 -->
On Apr 17 the VRB in b0eb12 slot 14 was replaced because of 
bit level errors. 
 
After that we seem to have gotten more of the converter node 
input errors which eventualy lead to dropping subfarms. 
 
Were able to reproduce the dropping subfarm problem when running 
without silicon and boosting the rates up by adjusting the  
prescales. 
 
There are front end errors pointing to b0eb12 and we reseated 
all the VRBs in that crate. 
 
After that the error rate seems to have gone away/been reduced. 
 
Running now without silicon and with high rates... to see if  
the problem shows up again. 
  
<!-- Author: Frank Chlebana -->


<!-- Date: Mon Apr 21 11:00:19.706 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3792&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3792' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3793&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3793' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3794&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3794' width='300'></a></center>
<!-- Author: Thorsten -->


<!-- Date: Mon Apr 21 11:00:22.145 2003 -->
Busy Timeout 
Event ID Error: 
0l3gate1.fnal.gov:main:10:56:28 AM->Host b0eb12.fnal.gov, task tRec_0 
SCPU-P0-E-TracerEventId: Event 536718, crate 6, channel 5 has either bad Tracer ID or bad markers around Tracer word. 
 Tracer word event ID is 0, should be 14. 
Tracer word = 0x04000006 (I think). 
Channel data = 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 ... 
... 0x00000000 0x00000000 0x00000000 0x00000000 0x02000000 0x05000000 0x04000006 0x00000000 -->   
 Additional Information:  
 
Attention!!!. SCPU_TRACER_EVENT_ID Error !!!  
 Hardware EVB has detected a problem with data quality in  
 SCPU b0eb12 (forwarded by FER crate CCAL_05).  
 Datataking is paused: events are not being processed.   
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797' target=_top>161797</a>) -->

<!-- Date: Mon Apr 21 11:03:48.002 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797">161797</A>
 Terminated at 2003.04.21 11:02:27
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 11:04:34.754 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797">161797</A>
TERMINATE: finished non-si physics run with changed prescales 
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 11:06:56.506 2003 -->
<pre>Spurious TriggerInhibits from not coordinating reset with bringing hv down.</pre>
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797' target=_top>161797</a>) -->


<!-- Date: Mon Apr 21 11:11:48.913 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161798">161798</A>
ACTIVATE: TOF QIE Calibration
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 11:11:49.884 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161798">161798</A>
TERMINATE: successful tof qie calib.
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 11:19:32.121 2003 -->
Shot setup of the HV, but plug (expert working on the plug)
<!-- Author: Thorsten -->


<!-- Date: Mon Apr 21 11:19:49.368 2003 -->
<pre>   We have been looking into the L3 converter dropout problem. 
 
The tests that we have done: 
  - reproduced the problem within 10-20min with the same run 
     configuration as used for physics data taking.  Subfarm c04 
     died. (no HRRs) 
  - removed SVX and the problem showed up again on subfarm c05 
     (no HRRs) 
  - reseated all VRBs in b0eb12, the problem reappeared in 
      40min or so after busy timeout and HRR. 
In the above tests we kept L2 output rates at maximum level 
(60-70 with SVX, 140 or so without SVX) by changing L2 prescale 
settings. 
  One of the changes to the system that happened on Thursday 
right before the problem appeared first is the replacement 
of the VRB in b0eb12 slot 14. That is why our 3d test above 
involves reseating the VRB cards. 
  At the moment Frank and Jim are starting on VRB tests for 
the VRBs in b0eb12. 
</pre>
<!-- Author: Ilya/Guillelmo/Steve/... -->


<!-- Date: Mon Apr 21 11:36:01.604 2003 -->
<p> 
I restarted iFix on the SVXIICON PC on the first floor to 
re-enable alarm logging (this was disabled when we upgraded to iFix 3.0 a few months ago --- whoops).  This will allow me to 
catch whatever is causing the spurious SVX/L00 alarms that have been occurring.  I already think I know what the source is, but 
the monitoring ace has been instructed to page the Si cooling pager next time it happens so I can confirm my suspicions.</p>
<!-- Author: Andy H. -->


<!-- Date: Mon Apr 21 11:48:26.442 2003 -->
<h3>New L1A rate limit</h3> 
Upped the Si L1A limit to 20 kHz  
<pre>Update: <UPDATE CARDS SET SILICONRATELIMIT='20000'</pre> 
now Greg has to take some action in the Trigger DB and it 
will become effective on the following COLDSTART
<!-- Author: SCN -->
<!-- Comment: Mon Apr 21 11;49;39 comment by...SCN -->
</pre> Something about pasting the Update message screwed up
the elog html.  Maybe this fixed it.


<!-- Comment: Mon Apr 21 11;49;50 comment by...SCN -->
</pre> Something about pasting the Update message screwed up
the elog html.  Maybe this fixed it.
<!-- Comment: Mon Apr 21 11;50;42 comment by...SCN -->
Comment went in twice because of simultaneous entry by Calorimeter people.
<!-- Comment: Mon Apr 21 11;51;20 comment by...Greg Feild -->
<pre>Action taken.  I added 2000 to each  of the four 
variables defining the allowed L1 accept rates.
</pre>
<!-- Date: Mon Apr 21 11:49:39.697 2003 -->
<b>Routine Weekly Calorimeter Crate Checkout</b><br> 
Over the weekend, pcal05 (and later, pcal01) had zero slopes for the calibration check.  This morning, that is true for all cal crates.  Since a normal part of the checkout is to reboot the crates, and there was a FER code change on friday, this seems to be the likely suspect.  <b><i>Besides the calibration failure, electronics look fine.</i></b>  Part of the code change involved the charge injection; this could cause the problems we are seeing.<p> 
We don't use charge injection during data taking, so I believe the cal crates are okay for data taking.  I will continue looking into this, and make more notes when I have something more solid than guesses.
<!-- Author: m mattson -->
<!-- Comment: Mon Apr 21 15;40;00 comment by...m mattson and z huang -->
Zhibin H and I believe the code bug has been caught.  The information has been forwarded to Frank C.  The next FER rebuild should allow the (QIE) calibrations to act normally (ie non-zero slopes).
<!-- Date: Mon Apr 21 11:51:27.043 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-3795&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-3795  width='300' align='top'></td><td>Shift luminosity for store 2445</a></td></tr></table>
<!-- Author: Thorsten -->

<!-- Date: Mon Apr 21 11:55:31.974 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161803">161803</A>
ACTIVATE: smx qie calibration
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 11:56:23.836 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161803">161803</A>
TERMINATE: successful smx qie calib
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 11:57:36.013 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161804">161804</A>
ACTIVATE: LED Calibration
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 11:58:47.809 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3796&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3796' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3797&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3797' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3798&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3798' width='300'></a></center>
<!-- Author: Thorsten -->

<!-- Date: Mon Apr 21 12:06:13.545 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161804">161804</A>
TERMINATE: LED calibration failed to get through activate state. No error but just waited
over 10' for ack's
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:06:14.729 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161805">161805</A>
ACTIVATE: LED Calibration
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:08:45.372 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161805">161805</A>
TERMINATE: Wrong Config for LED calib.
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:10:34.556 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161806">161806</A>
ACTIVATE: LED Calibration
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:12:16.533 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161806">161806</A>
TERMINATE: successful LED calibration
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 12:18:16.166 2003 -->
<pre>We ran the vrbtest to look for data errors in b0eb12 
vrb slots 10, 12 and 14. 
 
Did not see any errors. 
 
The hope is that after reseating/power cycling eb12 the 
errors were reduced/went away. 
 
Returning these crates to the shift crew...</pre>
<!-- Author: Frank and Jim -->

<!-- Date: Mon Apr 21 12:23:40.924 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161807">161807</A>
ACTIVATE: XeFlasher Calibration
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:24:51.484 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161807">161807</A>
TERMINATE: Finished Xenon Flasher Calibration
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:27:03.108 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161808">161808</A>
ACTIVATE: Muon Calibrations (WITH musc00)
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:29:17.199 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161808">161808</A>
TERMINATE: Successful Muon Calibrations
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:32:28.446 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161809">161809</A>
ACTIVATE: TOF TAC Calib
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:35:53.516 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161809">161809</A>
TERMINATE: finished TOF TAC calib
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 12:47:18.625 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161810">161810</A>
ACTIVATE: evb tests
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 13:12:01.378 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161812">161812</A>
ACTIVATE: L2 torute. NO COT, no CLC, no muon scill.
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 13:14:10.131 2003 -->
<pre>Final protons are being loaded now.</pre>
<!-- Author: mako -->
<!-- Comment: Mon Apr 21 13;19;24 comment by...mako -->
<pre>Trying again...</pre>


<!-- Date: Mon Apr 21 13:22:56.086 2003 -->
Lowered SL8 S1B another 50V (discharges were getting very large).  These wires will be masked off in the XFT.
<!-- Author: Aseet -->
<!-- Comment: Mon Apr 21 13;34;36 comment by...Cheng-Ju -->
<pre>We have updated the deadwire mask in the XFT to include
SL8 s1 cells 120-239 (quad B).</pre>
<!-- Date: Mon Apr 21 13:29:04.932 2003 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2003.04.21</td><td>13:25:26</td><td>W Inner BLM</td><td>0.61</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>13:25:26</td><td>W Outer BLM</td><td>0.05</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>13:25:26</td><td>E Inner BLM</td><td>2.41</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>13:25:26</td><td>E Outer BLM</td><td>8.18</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong>
<!-- Author: Thorsten -->


<!-- Date: Mon Apr 21 13:33:57.862 2003 -->
<pre>Antiprotons are being loaded now.</pre>
<!-- Author: mako -->
<!-- Comment: Mon Apr 21 13;48;18 comment by...mako -->
<pre>All antiprotons loaded.</pre>
<!-- Date: Mon Apr 21 13:46:21.623 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161815">161815</A>
TERMINATE: end fast rate run test of evb/le
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 13:54:38.141 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161816">161816</A>
ACTIVATE: Shot setup run, no si
<!-- Author: Steve2080 -->


<!-- Date: Mon Apr 21 13:55:57.584 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161816">161816</A>
TERMINATE: End of shot-setup run
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 14:14:27.684 2003 -->
<pre>Store 2447 scraping complete. 
 
Initial B0 Lum 39.76E30.</pre>
<!-- Author: mako -->

<!-- Date: Mon Apr 21 14:19:16.747 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817">161817</A>
 Activated at 2003.04.21 14:17:54
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 14:19:17.781 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817">161817</A>
ACTIVATE: Attempted Physics run with silicon but no dynamic prescaling
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 14:22:02.136 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3799&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3799' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3800&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3800' width='300'></a></center>
<!-- Author: Thorsten -->
<!-- Comment: Mon Apr 21 14;22;46 comment by...Thorsten -->
Losses and radiation during scraping
<!-- Date: Mon Apr 21 14:22:44.101 2003 -->
<pre>At current Lum of 38.4 (dynamic Prescaling DISABLED): 
 deadtime is 0.7% 
 L1A rate 8.2 kHz, 
 L2A 210 Hz 
 L3A at 59 Hz 
 CSL Input 15 MB/sec 
 CSL out 16.5MB/sec</pre>
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817' target=_top>161817</a>) -->



<!-- Date: Mon Apr 21 14:30:14.020 2003 -->
<pre>L3 Error messages return pointing at eb11. Subfarm drops out. 
Deadtime still low...</pre>
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817' target=_top>161817</a>) -->
<!-- Comment: Mon Apr 21 14;31;20 comment by...steve -->
Whoops, meant converter c11, not eb11 (!)

<!-- Comment: Mon Apr 21 14;35;06 comment by...steve -->
And another subfarm going down: b0l3c04

<!-- Comment: Mon Apr 21 14;46;59 comment by...steve -->
And now 03 is dying


<!-- Date: Mon Apr 21 14:52:29.507 2003 -->
cpr trip, cot trip, pause while recovering
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817' target=_top>161817</a>) -->


<!-- Date: Mon Apr 21 15:01:07.657 2003 -->
CPR( West 0,..5) and COT(SL8 S1b) tripped. 
<!-- Author: Thorsten :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817' target=_top>161817</a>) -->
<!-- Comment: Mon Apr 21 15;02;10 comment by...Thorsten -->
The same cot channel tripped again
<!-- Comment: Mon Apr 21 15;08;16 comment by...Thorsten -->
Scico paged expert
<!-- Date: Mon Apr 21 15:01:14.508 2003 -->
pause at another cot trip
<!-- Author: steve :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817' target=_top>161817</a>) -->




<!-- Date: Mon Apr 21 15:10:03.796 2003 -->
<pre>D0 informs us they are inserting Roman Pots.</pre>
<!-- Author: mako -->

<!-- Date: Mon Apr 21 15:16:03.357 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817">161817</A>
 Terminated at 2003.04.21 15:15:06
<!-- Author: RunControl -->



<!-- Date: Mon Apr 21 15:16:15.578 2003 -->
Lowering SL8-S1b 100V below nominal seems to have made things worse (compared to 50V below nominal).  Might indicate breakdown to neighboring sense lines.  Try an in-between setting: 75V below nominal.
<!-- Author: Aseet Mukherjee :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817' target=_top>161817</a>) -->

<!-- Date: Mon Apr 21 15:17:46.284 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817">161817</A>
TERMINATE: Ran with no dynamic prescale. 3 L3 subfarms died inside.
COT HV changed sl8 1b.
COT expert requests end of run 
<!-- Author: Steve2080 -->



<!-- Date: Mon Apr 21 15:21:44.253 2003 -->
<pre>Ended run on COT  HV changes. Took opportunity to 
reset and clean up event builder with goal of perhaps 
recovering some lost L3 converter nodes.</pre>
<!-- Author: steve -->


<!-- Date: Mon Apr 21 15:22:01.265 2003 -->
Another COT trip in SL8 S1b
<!-- Author: Thorsten :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817' target=_top>161817</a>) -->


<!-- Date: Mon Apr 21 15:33:11.290 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3801&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3801' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3802&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3802' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3803&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3803' width='300'></a></center>
<!-- Author: Thorsten -->
<!-- Comment: Mon Apr 21 15;33;48 comment by...Thorsten -->
The hourly plots



<!-- Date: Mon Apr 21 15:41:25.815 2003 -->
L3 Relay error on partitioning. Popup from errorlogger 
indicated directions which included starting to recover 
by hitting : 
do End/Abort+Reset ON ALL DEFINED PARTITIONS    
But, End is off-limits. so we hit reset and failed again  
<!-- Author: steve -->


<!-- Date: Mon Apr 21 15:54:52.914 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3804&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3804' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3805&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3805' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3806&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3806' width='300'></a></center>
<!-- Author: Thorsten -->

<!-- Date: Mon Apr 21 15:56:31.307 2003 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161792>161792</A> 
x27800</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_04 [6,271,360]</TD>                                       
<TD ALIGN=CENTER>07:16:46</TD>                                          
<TD ALIGN=CENTER>08:20:46</TD>                                          
<TD ALIGN=CENTER>00:57:34</TD>                                          
<TD ALIGN=RIGHT>     33,630,967</TD>                                    
<TD ALIGN=RIGHT>       409,097</TD>                                     
<TD ALIGN=RIGHT>     116,075</TD>                                       
<TD ALIGN=RIGHT>   51.318</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161795>161795</A> 
x27803</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_04 [6,271,360]</TD>                                       
<TD ALIGN=CENTER>08:57:29</TD>                                          
<TD ALIGN=CENTER>09:26:39</TD>                                          
<TD ALIGN=CENTER>00:27:30</TD>                                          
<TD ALIGN=RIGHT>     14,306,797</TD>                                    
<TD ALIGN=RIGHT>       218,733</TD>                                     
<TD ALIGN=RIGHT>     102,724</TD>                                       
<TD ALIGN=RIGHT>   22.934</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161796>161796</A> 
x27804</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_04 [6,271,360]</TD>                                       
<TD ALIGN=CENTER>09:39:57</TD>                                          
<TD ALIGN=CENTER>09:47:02</TD>                                          
<TD ALIGN=CENTER>00:06:40</TD>                                          
<TD ALIGN=RIGHT>      1,472,101</TD>                                    
<TD ALIGN=RIGHT>        63,171</TD>                                     
<TD ALIGN=RIGHT>      47,060</TD>                                       
<TD ALIGN=RIGHT>    5.388</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>0</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161797>161797</A> 
x27805</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_04 [6,271,360]</TD>                                       
<TD ALIGN=CENTER>10:03:28</TD>                                          
<TD ALIGN=CENTER>11:02:27</TD>                                          
<TD ALIGN=CENTER>00:52:11</TD>                                          
<TD ALIGN=RIGHT>     12,290,654</TD>                                    
<TD ALIGN=RIGHT>       593,708</TD>                                     
<TD ALIGN=RIGHT>     475,709</TD>                                       
<TD ALIGN=RIGHT>   41.188</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>0</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161817>161817</A> 
x27819</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_1_04 [6,271,360]</TD>                                       
<TD ALIGN=CENTER>14:17:54</TD>                                          
<TD ALIGN=CENTER>15:15:06</TD>                                          
<TD ALIGN=CENTER>00:31:58</TD>                                          
<TD ALIGN=RIGHT>     15,312,916</TD>                                    
<TD ALIGN=RIGHT>       393,300</TD>                                     
<TD ALIGN=RIGHT>     105,383</TD>                                       
<TD ALIGN=RIGHT>   70.805</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>15:55:01</TD>                                                       
<TD ALIGN=CENTER>02:55:55</TD>                                          
<TD ALIGN=RIGHT>      77,013,435</TD>                                   
<TD ALIGN=RIGHT>      1,678,009</TD>                                    
<TD ALIGN=RIGHT>       846,951</TD>                                     
<TD ALIGN=RIGHT>  191.634</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Mon Apr 21 15:56:45.501 2003 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2003.04.21</td><td>15:50:27</td><td>W Inner BLM</td><td>61.51</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>15:50:27</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>15:50:27</td><td>E Inner BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>15:50:27</td><td>E Outer BLM</td><td>61.51</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong> 
  
<!-- Author: Thorsten :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161818' target=_top>161818</a>) -->


<!-- Date: Mon Apr 21 16:00:12.586 2003 -->
<pre>3 successive failures to partition L3, each with an ensuing 
L3_Relay error. On last two fixes we also cleaned up L3. 
3 different phone calls with l3 experts, and 3 different 
flavors of L3_relay_errors, one of which referred to a 
suspect node (251). L3 expert recommended dropping associated 
subfarm (c15). 
This was done but we failed on partition again. Cleaned up 
L3 and tried successfully (at last) to partition. 
</pre>
<!-- Author: steve -->


<!-- Date: Mon Apr 21 16:01:25.880 2003 -->
<font color=red>Shift Summary: </font><pre>08:00 Store 2445 w/ B0 lum 14.3 
      Pbar stack: 134 mA + 6.5 mA/hour 
08:20 End run 161792. Persistent and unrecoverable 
      errors from various parts of the system. 
 
A couple of attempts to start a new run failed. 
Paged L3/EVB experts and let them do the cleanup. 
A couple of test runs to check on L3/EVB. 
 
11:15 End of Store 2445. 
 
Some non-QT calibrations. 
 
Other activities during shot setup: 
  Giovanni working on plug laser scans. 
  Aseet on COT. 
  Ilya, Guillelmo, Steve T. on EVB. 
  Frank C. and Jim P. on b0eb12. 
 
13:20 Loading final protons. 
13:34 Loading antiprotons. 
13:50 All antiprotons loaded. 
14:15 Scraping complete. Store 2447. 
      Initial B0 lum 39.76E30. 
 
14:20 Run 161817 w/ Si but dynamic prescaling 
      disabled. 
 
      Deadtime 0.7% 
      Trigger rates 8.2k/210/59 Hz 
      CSL in/out    15/16.5 MB/sec 
 
      Lost 3 L3 subfarms in 10/15/25 min. 
 
14:50 CPR and COT SL8 tripped at the same time. 
      COT tripped twice, so called Aseet. 
      Aseet adjusted voltage. 
 
      Another COT trip while we were ending run. 
 
      Meanwhile we were trying to recover the 
      3 dead subfarms and lost control of the 
      entire system; L3 now refuses to partition! 
 
16:00 We are waiting for L3/EVB experts to come in.</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   161792, 161817
Delivered Luminosity   383.8 nb-1  
Acquired Luminosity    153.6 nb-1  
Efficiency             40%
   
</td></tr></table></pre>
<!-- Author: mako -->

<!-- Date: Mon Apr 21 16:03:32.284 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161818">161818</A>
 Activated at 2003.04.21 16:02:08
<!-- Author: RunControl -->


<!-- Date: Mon Apr 21 16:03:33.959 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161818">161818</A>
ACTIVATE: PHYSICS_1_04
<!-- Author: Aidan Robson -->



<!-- Date: Mon Apr 21 16:06:08.874 2003 -->
 
2'35" b0dap63.fnal.gov:ConsumerErrorRe:4:03:14 PM->Runtime Error 1, Event 227, RunNum 161818: SvxMon Halt Recover Run: Pipeline out of synch in 135 silicon readout chips . 
Manual HRR; now enabled auto HRR
<!-- Author: Aidan Robson :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=161818' target=_top>161818</a>) -->


<!-- Date: Mon Apr 21 23:58:42.022 2003 -->
<center><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3830&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3830' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3831&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3831' width='300'></a><a href='/cgi/elog/elog.pl?nb=2003&action=view&page=-3832&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2003&action=view&page=-3832' width='300'></a></center>
<!-- Author: jaison -->


<!-- Date: Tue Apr 22 00:00:19.612 2003 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
shift losses 
 
<tr><td>2003.04.21</td><td>23:55:26</td><td>W Inner BLM</td><td>353.07</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>23:55:26</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>23:55:26</td><td>E Inner BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2003.04.21</td><td>23:55:26</td><td>E Outer BLM</td><td>352.40</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong>
<!-- Author: jaison -->


<!-- Date: Tue Apr 22 00:02:02.397 2003 -->
Begin shift or beam at 16:00:01<br> 
End shift or beam at 23:58:28<br> 
Delivered luminosity: 735.76<br> 
Acquired luminosity: 666.53<br> 
<hr> 
<TABLE bgcolor=#cdc9a5> 
<tr><th colspan=2>Totals</th></tr> 
<tr><td>Date:</td><td>2003.04.21</td> 
<tr><td>Shift:</td><td>eve</td> 
<tr><td>Delivered luminosity:</td><td>735.76 nb<sup>-1</sup></td></tr> 
<tr><td>Acquired luminosity:</td><td>666.53 nb<sup>-1</sup></td></tr> 
<tr><td>Efficiency:</td><td>90.59</td></tr> 
</TABLE>  
<!-- Author: jaison -->


<!-- Date: Tue Apr 22 00:02:30.540 2003 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2003&action=view&page=-3833&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2003&action=view&page=-3833  width='300' align='top'></td><td>.</a></td></tr></table>
<!-- Author: . -->
<!-- Comment: Tue Apr 22 00;02;54 comment by...jaison -->
shift lum.