<!-- Title: Eve shift. Wed Nov  7, 2001 -->
<!-- SciCo: NONE -->
<!-- DAQAce: Koji/Denys -->
<!-- MonAce: Koji/Denys -->
<!-- CO: NONE -->
<!-- OpManager: Dave Ambrose -->
<!-- Notes: -->
<pre></pre>


<!-- Date: Wed Nov 7 16:26:56 2001 -->
<pre>Returned Level3 after testing a new code.   
Siwtched back to the default version. </pre>
<!-- Author: Boris.  -->
<!-- Comment: Wed Nov  7 17;02;55 comment by...Boris.  -->
<pre>We tested a new version of Level3 Control code from 
Jeff Tseng in attempt to fix minor bugs.
Jeff made some progress debugging the probem. 
We will need to do more tests in the future. 



</pre>


<!-- Date: Wed Nov 7 17:04:58 2001 -->
Nikolai is downstairs to fix HV problem of north central 
calorimeter. 
<BR> 
He's also going to shut down CCAL crates.  
<!-- Author: Koji/Denys -->


<!-- Date: Wed Nov 7 17:08:05 2001 -->
Before Nikolai came in, we tried to start SMX calibration and  
PCAL05 didn't respond in partitioning. 
<BR> 
We cannot ping it, vxlogin, reboot through vxworks or minicom. 
<BR> 
Paged Robin. 
<BR> 
We checked that PSM alarm in iFix and it looks as if the power  
supply has tripped. 
<BR> 
Robin went downstairs to fix this.  
<!-- Author: Koji/Denys -->


<!-- Date: Wed Nov 7 17:09:21 2001 -->
Steve Tether starts his work on HardEVB. 
<BR> 
Until he's finished, we cannot use it at all.
<!-- Author: Koji/Denys -->
<!-- Comment: Wed Nov  7 17;12;52 comment by...Steve Tether -->
I've physically disconnected b0l3pcom1 and b0l3pcom2 from their networks because I found a directory /.old on pcom1 which I can't account for. I'm not sure it's a break-in yet, but the directory contains copies of /dev and /etc so I'm playing it safe. I did not find /.old on b0l3pcom2.
<!-- Comment: Wed Nov  7 20;01;53 comment by...Steve Tether -->
It looks like a break-in via the Kerberos ftpd (which I THOUGHT we had disabled). It appears that no damage was done because we stopped the hackers when we rebooted to change the IP numbers of the gateway nodes). Tomorrow morning I'll start bringing the gateways back on-line.
<!-- Date: Wed Nov 7 17:10:42 2001 -->
JC is here working on iFix.  
<!-- Author: Koji/Denys -->


<!-- Date: Wed Nov 7 17:12:44 2001 -->
Robin is back from downstairs, says that she didn't understand 
what caused the power outage. 
<BR> 
Anyway, she turned the power on again. 
<BR> 
We see pcal05 is green again on VxWorks.
<!-- Author: Koji/Denys/Robin -->
<!-- Comment: Wed Nov  7 17;24;54 comment by...Koji/Denys -->
iFix for this power supply (PSM-EPNW_3) also comes back to the
 usual status (it's still alarming, but it's a know problem).
<!-- Date: Wed Nov 7 17:15:18 2001 -->
JC finished his work and iFix is its usual operation mode now.  
<!-- Author: Koji/Denys -->



<!-- Date: Wed Nov 7 17:35:45 2001 -->
About fifteen minutes ago, Kevin Lannen finished his work  
with CMX crates.
<!-- Author: Koji/Denys -->


<!-- Date: Wed Nov 7 18:05:58 2001 -->
We noticed that CMX_01 is not responding. 
<BR> 
We cannot do anything (ping, reboot or reset) with this  
crate.  Let the experts (Abraham) know.
<!-- Author: Koji/Denys -->


<!-- Date: Wed Nov 7 19:22:29 2001 -->
Nikolai finished his work.  
<!-- Author: Koji/Denys -->


<!-- Date: Wed Nov 7 19:48:22 2001 -->
As we were in 'setup' transition of SMX QIE calibration,  
we got an error : 
<PRE>Partition:4 Run number: 131208 (0x20088) 
(MLE) b0ccal01:Messenger:7:41:57 PM->ID PROM check failure in slot 2 (ASCII blan 
k) 
(MLE) b0ccal01:Messenger:7:41:57 PM->No connection to tracer 
(MLE) b0ccal01:Messenger:7:41:57 PM->VISIONread/write failed in slot 2 
(MLE) b0ccal01:Messenger:7:41:57 PM->Statechange failed 
</PRE> 
Paging SMXR expert.
<!-- Author: Koji/Denys :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131208' target=_top>131208</a>) -->


<!-- Date: Wed Nov 7 19:52:08 2001 -->
We just did TOF TAC calibration, there were no errors.
<!-- Author: denys/koji :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131210' target=_top>131210</a>) -->


<!-- Date: Wed Nov 7 19:57:41 2001 -->
Cal QIE also fails. 
<PRE>Partition:4 Run number: 131211 (0x2008b) 
(MLE) b0ccal01:Messenger:7:55:05 PM->ID PROM check failure in slot 2 (ASCII blan 
k) 
(MLE) b0ccal01:Messenger:7:55:05 PM->No connection to tracer 
(MLE) b0ccal01:Messenger:7:55:05 PM->VISIONread/write failed in slot 2 
(MLE) b0ccal01:Messenger:7:55:06 PM->Statechange failed 
</PRE> 
Paging ADMEM expert.
<!-- Author: Koji/Denys :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131211' target=_top>131211</a>) -->
<!-- Comment: Wed Nov  7 20;17;19 comment by...Koji -->
Tried rebooting ccal01 with little hope, still cannot start
calibration.
<!-- Date: Wed Nov 7 20:11:39 2001 -->
Steve Tether finished his work. 
<BR> 
Hardware event builder will be kept down till tomorrow, 
so we'll be working with softEVB.  
<!-- Author: Koji/Denys -->



<!-- Date: Wed Nov 7 20:35:47 2001 -->
I checked the ominous TRACER in b0ccal01, it appears to be 
completely dead (inaccessible through VME), also after a VME reset. Robin agreed also 
to exclude it from all activities for tonight, they'll have a 
closer look tomorrow. 
<P> 
I also reset half the reset crate (b0res01) - it was (again) 
hanging.  
<!-- Author: Arnd -->


<!-- Date: Wed Nov 7 21:11:41 2001 -->
Tring to take cal QIE calibration. 
<BR> 
We can go through the RC diagram but calibration comes to  
DONE status 'No Events yet'. 
<BR> 
After hitting 'end' anyway, the transition doesn't finish. 
<BR>
<!-- Author: Koji :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131216' target=_top>131216</a>) -->


<!-- Date: Wed Nov 7 21:12:31 2001 -->
Released WCAL crates to Youngdo. He'll finish in 10 min or so.
<!-- Author: Koji -->


<!-- Date: Wed Nov 7 22:28:49 2001 -->
As we tried several things like changing partition and  
run configurations, we noticed in RC error log : 
<PRE> 
b0dap62.fnal.gov :  sevb : 9:43:00 PM 
 -> Problem on CSL connection.  Status returned: 99 
</PRE> 
We restarted CSL with Tony, who replied our paging, on the phone. 
After restarting CSL, we took SMQIE calibration without CCAL01. 
<BR> 
I was deceived by the 'No event yet' in event number column of RC, 
but it's in the database and we can look at the data with DBANA.
<!-- Author: Koji/Denys :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131231' target=_top>131231</a>) -->
<!-- Comment: Wed Nov  7 22;34;12 comment by...Koji/Denys -->
SORRY. Run number for this SMX calib is 131230.


<!-- Date: Wed Nov 7 22:37:41 2001 -->
<pre>just took CAL QIE calibration. Did it without b0ccal01 
since that crate has some problems. 
Calibration worked but we got the following errors 
in error logger: 
 
 
CT: 11/7/2001, 23:23'41" 
Partition:4 Run number: 131231 (0x2009f) 
(RC)   23:23:50 Config -&gt; READY 
(RC)   23:24:1 Activate -&gt; CALIBRATING 
(b0wcal02) QIE calib: slot 18, qie 04:Warn  4: Raw ped RMS high   , val 05.1183, cap 0, rng0. 
(MLE) b0puls01:Messenger:10:24:21 PM-&gt;QIE calibration failed     
  (MLE) b0puls01:Messenger:10:24:21 PM-&gt;QIE calibration failed 
(MLE) b0puls01:Readout_Calib:10:24:21 PM-&gt;ADMEM=21 QIE=0 Fail 10: No fitting          (0.0) (0) (0) 
(MLE) b0wcal00:Readout_Calib:10:24:20 PM-&gt;ADMEM=18 QIE=6 Fail  6: Raw ped RMS high    (22.000181) (0) (0) 
(MLE) b0ccal04:Readout_Calib:10:24:27 PM-&gt;ADMEM=16 QIE=17 Fail 14: Sigma(off) big      (2.4550953) (1) (0) 
(MLE) b0ccal00:Readout_Calib:10:24:26 PM-&gt;ADMEM=20 QIE=10 Fail 14: Sigma(off) big      (2.3387673) (1) (1) 
(MLE) b0ccal14:Readout_Calib:10:24:26 PM-&gt;ADMEM=19 QIE=4 Fail 14: Sigma(off) big      (1.4683257) (3) (1) 
(MLE) b0pcal00:Readout_Calib:10:24:30 PM-&gt;ADMEM=17 QIE=12 Fail  6: Raw ped RMS high    (21.091703) (0) (0) 
(MLE) b0pcal11:Readout_Calib:10:24:33 PM-&gt;ADMEM=17 QIE=3 Fail 14: Sigma(off) big      (1.0609431) (0) (0) 
                              </pre>
<!-- Author: denys/koji :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131231' target=_top>131231</a>) -->


<!-- Date: Wed Nov 7 22:45:45 2001 -->
we did CAL Led calibration. 
No errors seen. 
DBANA plots look OK (as far as we can tell)
<!-- Author: denys/Koji :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131232' target=_top>131232</a>) -->


<!-- Date: Wed Nov 7 22:58:55 2001 -->
did CAL XEF calibration.  
no errors
<!-- Author: denys/koji :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131233' target=_top>131233</a>) -->


<!-- Date: Wed Nov 7 23:45:06 2001 -->
Started Rick's run although we cannot run with ccal01. 
<BR> 
Taking data with a rate of Fred L1 160Hz and L1 5Hz. 
<BR> 
Noticed that CEM and CHA were warning, Konstantin told us that 
Nikolai is developing software and until this task gets done 
we should ignore the warning.	
<!-- Author: Koji/Denys :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131234' target=_top>131234</a>) -->


<!-- Date: Wed Nov 7 23:53:59 2001 -->
we were trying to take the rick's run for the last hour but 
the data taking is very slow. In fact we only got 10k events 
during this hour. Since b0ccal01 is missing (which makes this 
run data completely useless according to Rick Tesarek) and shift is ending we decided it was more important to start taking cosmic 
data sooner.<br> 
So we are ending rick's run at 10795 events. Run number was 
131234, avg Fred L1A rate was 156.8 Hz.
<!-- Author: denys/koji :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131234' target=_top>131234</a>) -->


<!-- Date: Thu Nov 8 00:01:30 2001 -->
We started overnight cosmics run. 
all COT crates are out, ccal01 is out, cmx01 is out. 
Got error message about IMU_00 missing cards in slots 3,4,20 
which is a known problem. 
<br> 
Run has started successfully, we enabled auto recovery.  
<!-- Author: denys/koji :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=131234' target=_top>131234</a>) -->
<!-- Comment: Thu Nov  8 00;03;07 comment by...Denys/koji -->
<table border=0><td><tr><td bgcolor=#e0e030>Sorry, run number is 131235!</td></tr></td></table>


<!-- Date: Thu Nov 8 00:21:38 2001 -->
Run 131235 at event 5600. 
<BR> 
Rate : FRED 17Hz, L1 6Hz. 
<BR> 
TrigMon and YMon are running. 
<BR> 
Auto recovery of RC looks working all right. 
<font color=red><cite>(entry outside this shift's time range )</cite></font>
<!-- Author: Koji/Denys -->


<!-- Date: Thu Nov 8 00:29:04 2001 -->
Just as I was leaving, Konstantin showed up, told me that there are several channels which have hardware problems and Nikolai 
is working on that. Probably Nikolai has not yet finished this  
job. 
<BR> 
So there might be some channels with wrong HV settings.<font color=red><cite>(entry outside this shift's time range )</cite></font>
<!-- Author: Koji -->


<!-- Date: Thu Nov 8 00:31:42 2001 -->
<font color=red>Shift Summary: </font><pre><pre> - Several expert works done (L3 testing, hardEVB, iFix,  
   and accesses). 
 - PCAL05 suffered from power supply problem. got fixed. 
 - Tracer error in CCAL01 after access.  This will be fixed 
   tomorrow. 
 - HardEVB is kept down until some time tomorrow. worked with 
   softEVB. 
 - Had to restart CSL. 
 - TOF TAC, CALQIE, SMQIE, CALLED and CALXFE calibrations  
   taken without CCAL01. 
 - Discarded Rick's run at 10k events since we cannot include 
   CCAL01. 
 - Started overnight cosmics run with CMX01, CCAL01 and COT out. 
   Several channels with wrong HV setting. 
 - My last shift is over!! Good night. (Koji)</pre></pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   131208 - 131235
Delivered Luminosity   0  
Acquired Luminosity    0  
Efficiency             0
   
</td></tr></table></pre>
<!-- Author: Koji/Denys -->