<!-- Title: Day shift. Wed May 15, 2002 -->
<!-- SciCo: David Gerdes -->
<!-- DAQAce: Turgun Nigmanov -->
<!-- MonAce: Dmitry Tsybychev -->
<!-- CO: Richard Wigmans -->
<!-- OpManager: Bill Orejudos -->
<!-- Notes: -->
<pre>Tevatron studies scheduled to continue through end of this shift, with shot setup to follow on evening shift.  
 
Goals for this shift: perform calibrations, give detector to experts as needed, exercise the system.</pre>

<!-- Date: Wed May 15 08:30:18 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144832">144832</A>
 Terminated at 2002.05.15 08:29:49
<!-- Author: RunControl -->



<!-- Date: Wed May 15 08:39:28 2002 -->
COT calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144833' target=_top>144833</a>) -->


<!-- Date: Wed May 15 08:45:34 2002 -->
MUON calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144834' target=_top>144834</a>) -->


<!-- Date: Wed May 15 08:45:38 2002 -->
Rick Tesarek requests a series of special runs to measure PMT spikes. These runs must be taken during quiet time. We need quiet time for standard calibrations as well, so I called MCR to request an hour before the start of shot setup. No quiet time is currently planned, but they will consider our request.
<!-- Author: David Gerdes -->


<!-- Date: Wed May 15 08:49:56 2002 -->
TOF_TAC calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144835' target=_top>144835</a>) -->


<!-- Date: Wed May 15 08:55:53 2002 -->
TOF QIE calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144836' target=_top>144836</a>) -->


<!-- Date: Wed May 15 09:11:07 2002 -->
MINIPLUG calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144838' target=_top>144838</a>) -->


<!-- Date: Wed May 15 09:15:46 2002 -->
RP calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144839' target=_top>144839</a>) -->


<!-- Date: Wed May 15 09:32:26 2002 -->
Trying to torture the Silicon, I found a <B>SUSPENDED MESSENGER on b0clc00</b> as it would nto partition <pre> 
b0clc00-> i 
 
  NAME        ENTRY       TID    PRI   STATUS      PC       SP     ERRNO  DELAY 
---------- ------------ -------- --- ---------- -------- -------- ------- ----- 
tExcTask   excTask        efe738   0 PEND         29a268   efe660       0     0 
tNetTask   netTask        edd0b8   1 PEND         280574   edd008       0     0 
tPortmapd  portmapd       ed8338   1 PEND         280574   ed81c8      16     0 
tRlogOutTasrlogOutTask    dcfb40   2 PEND         280574   dcf9b8       0     0 
tRlogInTaskrlogInTask     dce5b0   2 PEND         280574   dce388       0     0 
tShell     shell          de7fc8   5 READY        283f38   de6f90   30065     0 
tRlogind   rlogind        ed98c8  15 PEND         280574   ed9508       0     0 
tAioIoTask1aioIoTask      eee6a8  50 PEND         2809e8   eee600       0     0 
tAioIoTask0aioIoTask      ee74a0  50 PEND         2809e8   ee73f8       0     0 
tAioWait   aioWaitTask    ef58b0  51 PEND         280574   ef5758       0     0 
rshelld    eba658         ebf908 100 PEND         280574   ebf7a0 ffffffff     0 
t1         VISIONserver   eb9878 100 PEND         280574   eb9748       0     0 
t2         ROBINserver    eb4850 100 PEND         280574   eb4710       0     0 
t3         httpDaemon     eaf828 100 PEND         280574   eaf610       0     0 
receive    clcreceive     ea0f98 200 PEND+T       280574   ea0b50      46   532 
Messenger  FER_messenge   e4cdf8 200 SUSPEND      d7f650   e4c9e8  3d0001     0 
rtlm_main  rtlm_main      eaa800 220 PEND         280574   eaa6c0       0     0 
Mon_III    FER_monitorI   dda178 220 DELAY        2834ec   dda0c8       0    33 
rtlm_sessiortlm_session   df2750 225 PEND         280574   df2570       0     0 
tLogTask   logTask        efbdc0 250 PEND         29a268   efbcf8       0     0 
value = 0 = 0x0 
b0clc00-> tt Messenger 
288aec vxTaskEntry    +60 : FER_messenger () 
d52430 FER_messenger  +2c0: d33574 () 
d33574 FER_smartInitMS+185c: 1ea2f0 () 
1ea2f0 TipcSrvMainLoop+510: TipcSrvMsgProcess (dba188) 
1e9dc0 TipcSrvMsgProcess+d8 : 1e7098 (fff210, dba188) 
1e73bc TipcSrvSubjectCbCreate+8c0: TutCbListCall (da7b98, fff210, e4cba8) 
1098e8 TutCbListCall  +158: d38d18 () 
d38dac FER_processReadoutListCallout+363c: d365d0 () 
d365d0 FER_processReadoutListCallout+e60: FER_coldstart () 
d58a0c FER_coldstart  +74 : d599b0 () 
d599b0 FER_init       +e00: ADMEM_initL1Trig () 
value = 0 = 0x0 
</pre> 
Actions<ul> 
<li> rebooted it ("reboot" from VxWorks prompt) but it doesn't 
seem to want to come back to life. (Nothing from vxlogin or minicom) 
<li>  Hit the reset button on the crate-now it partitions 
<li> Also it survives Cold Start (but the L3 Filters did not...) 
</ul> 
I guess it is ok now
<!-- Author: SCN -->


<!-- Date: Wed May 15 09:36:24 2002 -->
Still trying to torture Silicon, using  
<I>L2 TORTURE 4,1,290</I> but <B>all the filters died (Run 144840)</B>. 
Now trying <I>L2 TORTURE 4,1,292</I>...
<!-- Author: SCN :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144840' target=_top>144840</a>) -->


<!-- Date: Wed May 15 09:43:33 2002 -->
<I>L2 TORTURE 4,1,292</I> worked.  Got some RF errors  
out of run 144841 but <B>client svtmon perished in the HALT transition</B> so I removed it.
<!-- Author: SCN :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144841' target=_top>144841</a>) -->


<!-- Date: Wed May 15 10:20:09 2002 -->
Got running at  ~10 kHz L1A but did not see RF rate go through the roof.  Tried to increase L1A rate but I'm not experienced 
enough to effectively play with the prescales. 
One consequence of small L2 rate is that "instantaneous 
RF rate" has large fluctuations due to relatively small 
(like 1 or 2) increases in total # of RF errors.  Anyway, releasing crates for calibration etc.  Also had 
some <B>Partition Error (XPT)</B> at the end of the run.  Don't 
know what that means.
<!-- Author: SCN :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144843' target=_top>144843</a>) -->


<!-- Date: Wed May 15 10:23:31 2002 -->
<pre>Calorimeter Qie Calibrations</pre>
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144844' target=_top>144844</a>) -->
<!-- Comment: Wed May 15 10;26;10 comment by...Turgun -->
Error during CAL_QIE calibration:
(MLE) b0ccal00:Readout_Calib:10:18:12 AM->ADMEM=19 QIE=2 Fail 14: Sigma(off) big      (1.3256739) (1) (1)



<!-- Date: Wed May 15 10:26:26 2002 -->
For reasons beyond my understanding, somebody had removed the 
miniplug crates (fcal00/01) from an important boot script, so 
that they would not boot anymore?!??! I fixed this, and I note 
that we were very lucky to find this before the next shot.<P> 
Since I want all front-end crates to be rebooted anyways, we'll 
see if other problems show up. 
  
<!-- Author: Arnd -->


<!-- Date: Wed May 15 10:38:34 2002 -->
SMX cxalibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144845' target=_top>144845</a>) -->


<!-- Date: Wed May 15 10:39:03 2002 -->
LED calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144846' target=_top>144846</a>) -->


<!-- Date: Wed May 15 10:41:22 2002 -->
Xenon calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144847' target=_top>144847</a>) -->


<!-- Date: Wed May 15 11:04:51 2002 -->
Plug Laser calibration is done
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144848' target=_top>144848</a>) -->


<!-- Date: Wed May 15 11:11:00 2002 -->
All front-end crates with readout have been rebooted to pick up 
new code, with added diagnostics and timing measurements. It is 
important to take runs with the full system (no Silicon required) 
well before the next store goes in, so that we can switch back 
in case of problems (crates crashing etc.).  
<!-- Author: Arnd -->


<!-- Date: Wed May 15 11:15:40 2002 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2002&action=view&page=-3041&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2002&action=view&page=-3041  width='300' align='top'></td><td>Roman Pots on May 7 19:40-20:40</a></td></tr></table>
<!-- Author: dmitri -->


<!-- Date: Wed May 15 11:17:26 2002 -->
During the CLC calibration it terminated with error: 
 
(MLE) b0clc00:Readout_Calib:11:01:08 AM->ADMEM=10 QIE=2 Fail  6: Raw ped RMS high    (20.310465) (0) (0) 
(b0clc00) QIE calib: slot 10, qie 02:Fail  6: Raw ped RMS high   , val 20.3105, cap 0, rng0.  
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144851' target=_top>144851</a>) -->


<!-- Date: Wed May 15 11:30:55 2002 -->
BSC calibration failed due to b0clc00 crate was not able to use.
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144849' target=_top>144849</a>) -->


<!-- Date: Wed May 15 11:57:25 2002 -->
Steven Vejcik got all CCAL  and CMU crates for some tests, ~2 hours.
<!-- Author: Turgun -->


<!-- Date: Wed May 15 12:00:20 2002 -->
We are running with default trigger table without the beam on TEV
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144853' target=_top>144853</a>) -->


<!-- Date: Wed May 15 12:15:56 2002 -->
<pre>reboot ccal,cmu with test front end code for cmu calib.</pre>
<!-- Author: vejcik -->


<!-- Date: Wed May 15 12:18:43 2002 -->
EVB TO error. Can't login b0eb18, 19, 22 
Ilya is here and looking it.
<!-- Author: Turgun -->
<!-- Comment: Wed May 15 12;31;56 comment by...Ilya -->
<pre>   These SCPUs hung. I had to push reset buttons on them.</pre>


<!-- Date: Wed May 15 12:33:54 2002 -->
<br><b>BSC QIE Calibration:</b> 
<br>Fixed the code preventing a BSC QIE calibration and performed 
such a calibration.  The results of the calibration should be  
in the DB. 
<br>
<!-- Author: Arnd Meyera, Rick Tesarek :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144856' target=_top>144856</a>) -->


<!-- Date: Wed May 15 12:40:54 2002 -->
<pre>Cleanup EVB and resetting some crates. Start run without beam. 
Default trigger table. CCAL crates used by S.Vejcik. 
FCAL00, 01 and CMU00,01 are used by Rick.</pre>
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144857' target=_top>144857</a>) -->


<!-- Date: Wed May 15 12:47:17 2002 -->
<pre>As a test I removed references to  
 
http://www-b0.fnal.gov:8000 
 
in the ace2help index.html file. 
 
This makes references relative to the web server... and 
is something that we will have to do as part of the migration 
to the dedicated web server...</pre>
<!-- Author: Frank Chlebana -->


<!-- Date: Wed May 15 12:49:01 2002 -->
<br><b>Forward QIE calibration:</b> 
<br>After fixing the code (noted above) took another QIE  
calibration including the miniplug crates (b0fcal00, 01). 
This QIE calibration was also successful.  The 
miniplug trigger thresholds probably need to be adjusted in  
the DB.  
<br>
<!-- Author: Arnd Meyer, Rick Tesarek :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144858' target=_top>144858</a>) -->


<!-- Date: Wed May 15 12:58:14 2002 -->
<pre>CLC calibration is done. The message from errorlog: 
(MLE) b0clc00:Readout_Calib:12:55:24 PM->ADMEM=10 QIE=2 Fail  6: Raw ped RMS high    (20.221716) (0) (0) 
(b0clc00) QIE calib: slot 10, qie 02:Fail  6: Raw ped RMS high   , val 20.2217, cap 0, rng0. 
</pre>
<!-- Author: Turgun :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144859' target=_top>144859</a>) -->


<!-- Date: Wed May 15 13:20:36 2002 -->
<pre>Getting odd messages related to smart sockets on ccal crates. 
Although the FER being run is a private version, the messages  
might be noteworthy and I include them here: 
At boot: 
Done executing startup script boot/front-end/startup.all.flash 
b0ccal00-> Using local rtlm 
Azp TipcSrvLmCheck 
dummy add 107db0; tipcadd 1e5914, newins 7ff2249c 
Connecting to <tpc:localhost> RTlm. 
Using tcp protocol. 
received connection from 127.0.0.1:1025, socket is 15 
Message from RTlm: Connection established. 
WARNING: Internal validation failed, file: ipcmsg.c, line 1314WAQRNING: Internal 
validation failed, file ipcmsg.c, line 1314Message from RTlm: 
. 
. 
. 
Subsequently at startup, (within vmeBusScan) 
*** FATAL ERROR *** 
0xT_REALLOC <((msg->variable_data_buf).data)>: 67108864 bytes: out of heap: file ipcmsg1.c, line 981 
. 
. 
. 
 
</pre>
<!-- Author: vejcik -->
<!-- Comment: Wed May 15 13;25;37 comment by...sv -->
Also noted that if devel fer code is used, same messages
occur at boot (concerning validation within ipcmsg.c)
but not during bus scan at setup.



<!-- Comment: Wed May 15 14;44;13 comment by...Arnd -->
The "validation failed" messages are OK, and nothing to worry
about. The vmeBusscan message is not, but most likely to the
version of the code being used, as note by Steve.

<!-- Date: Wed May 15 13:51:46 2002 -->
Give back system for quiet time calib.  
<!-- Author: sv -->

<!-- Date: Wed May 15 14:02:35 2002 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144862">144862</A>
 Activated at 2002.05.15 14:00:48
<!-- Author: RunControl -->



Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=144862">144862</A>
 Terminated at 2002.05.15 14:01:43
<!-- Author: RunControl -->





<!-- Comment: Wed May 15 14;51;33 comment by...Turgun -->
special run for SUMET_01 study


<!-- Date: Wed May 15 15:05:02 2002 -->
There are L3 farm problem. We cannot to run Rick's special 
SUMET_01 studies. Experts are here.
<!-- Author: Turgun -->
<!-- Comment: Wed May 15 16;26;20 comment by...Ilya -->
<pre>   These are L3 manager problem related to creating
calibration tarballs for the NULL executable. This
causes all L3 filters to crash during the ColdStart.
Farrukh is looking into it.</pre>
<!-- Date: Wed May 15 15:46:56 2002 -->
Plug crates have been rebooted; this was the last batch that 
needed to pick up new front-end code.  
<!-- Author: Arnd -->


<!-- Date: Wed May 15 15:53:04 2002 -->
<table border=0><td><tr><td bgcolor=#e0e030>Plan from run coordinator: 36x4 store when stack reaches 70 mA (which it is right now). Take 10 mA off the stack for this, then continue stacking during Tevatron operation. Following this test, 
go into shot setup when stack reaches 80 mA. Estimate shot setup around 9PM.</td></tr></td></table>
<!-- Author: David Gerdes -->

<!-- Date: Wed May 15 15:55:10 2002 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>15:55:01</TD>                                                       
<TD ALIGN=CENTER>::</TD>                                                
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Wed May 15 16:00:41 2002 -->
<font color=red>Shift Summary: </font><pre>Continued Tevatron studies throughout this shift. We performed a complete set of calibrations. 
 
We encountered L3&lt;--&gt;database problems when attempting to do special SUMET runs for Rick Tesarek. Experts are here to investigate. This only effects null L3 exes, not physics tables. 
 
There will be a 36x4 store shortly, followed by shot setup late in the evening shift, we hope. 
 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   
Delivered Luminosity   0  
Acquired Luminosity    0  
Efficiency             
   
</td></tr></table></pre>
<!-- Author: David Gerdes -->