<!-- Title: Eve shift. Sat Feb  7, 2004 -->
<!-- SciCo: Pat Lukens -->
<!-- DAQAce: Vadim Khotilovich -->
<!-- MonAce: Natasha Miladinovic -->
<!-- CO: Olga Noriella -->
<!-- OpManager: JJ Schmidt -->
<!-- Notes: -->
<pre>We inherit smooth data taking - without the Time of Flight -  
Experts are still working on its high voltage. 
Acc. Div having troubles with a rad detector, so stacking has just started.</pre>


<!-- Date: Sat Feb 7 16:24:43.796 2004 -->
Done timeout from pcal08. Shepherded it and also pcal07 which we inherited in red condition from the previous shift. 
 
(MLE) b0l3pcom1.fnal.gov:main:4:11:46 PM->Host b0eb16.fnal.gov, task tRec_0 
SCPU-P1-E-VrbReadFailed: Error reading VRB in slot 12 for event 12712984. 
The VRB had no event. 
(MLE) b0dap73.fnal.gov:Thread-36816:4:11:50 PM->Done Timeout: PCAL_08  
(MLE) b0dap73.fnal.gov:Thread-36816:4:11:50 PM->Requested Halt-Recover-Run issued [errmon] 
CT: 2004.02.07 16:11:59 
11'59" 1 crate/s: b0pcal08(42),  Done TO.[RXPT]
<!-- Author: Vadim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881' target=_top>178881</a>) -->

<!-- Date: Sat Feb 7 16:26:29.753 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881">178881</A>
ACTIVE: Busy Condition in VRB/EVB forwarded by crates:
VRB_ISL_06 Busy conditions are produced by the VRB and are forwarded
 by the front end crates to the busy crosspoint.  Busy
 conditions do not represent a problem with the front end
 crates themselves, but with the EVB system.
-- HRR recovered it.
<!-- Author: Vadim x2080 -->



<!-- Date: Sat Feb 7 16:41:36.014 2004 -->
<pre>Still not stacking and no stack. 
JJ's stack chart.. 
 
0 - 125  16 hours 
0 - 160  24 hours 
 
</pre>
<!-- Author: jj -->
<!-- Comment: Sat Feb  7 17;07;14 comment by...JJ -->
<pre>Patrick Thomas Lukens complains that Eugene Edward Schmidt, Jr.
writes confusing elog entries. JJ accepts criticism and tries
again.

The Anti-Proton Source is still not producing pbars.
The accumulator has no current stack of pbars.

JJ's estimate for time to stack from 0 to 125 ma
is 16 hours. JJ's estimate for time to stack from
0 to 160 ma is 24 hours. 


</pre>
<!-- Date: Sat Feb 7 17:03:52.219 2004 -->
The expert for CEM, CHA and WHA came and informed us of a problem that he sees (bad first channel in all 3 systems). He believes it's a problem with the readout program, and not with the detectors themselves; therefore he'll kill and restart the program. This should not interfere with datataking. The expert will update us in about half an hour.
<!-- Author: natasha -->


<!-- Date: Sat Feb 7 17:10:48.440 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881">178881</A>
ACTIVE: "SCPU_BAD_VRB_BYTE_COUNT Error !!! Hardware EVB has detected a problem with data quality in SCPU b0eb16." - HRR recovered it
<!-- Author: Vadim x2080 -->



<!-- Date: Sat Feb 7 17:13:23.693 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2946&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2946' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2947&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2947' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2948&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2948' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2949&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2949' width='300'></a></center>
<!-- Author: natasha -->


<!-- Date: Sat Feb 7 17:47:23.450 2004 -->
There is still no stack, or any stacking.  We anticipate that this store will run well into tomorrow. We contacted the silicon group (main pager - J. Thom) to put them on notice that tomorrow morning should be a good time for the bias scan study.  They will plan accordingly, and call in the morning before coming in.
<!-- Author: Lukens -->


<!-- Date: Sat Feb 7 18:25:06.131 2004 -->
We had TOF00 and TOFHV ihibits masked but TOF01 and TOF02 not. They were just giving a lot of trigger inhibits and we masked them (people are working on TOF)
<!-- Author: Vadim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881' target=_top>178881</a>) -->

<!-- Date: Sat Feb 7 18:27:40.342 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881">178881</A>
 Terminated at 2004.02.07 18:27:15
<!-- Author: RunControl -->


<!-- Date: Sat Feb 7 18:28:40.151 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881">178881</A>
TERMINATE: Stopping due to the problem with central cal HV. HV was reset.
<!-- Author: Vadim x2080 -->


<!-- Date: Sat Feb 7 18:31:51.691 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178882">178882</A>
 Activated at 2004.02.07 18:31:40
<!-- Author: RunControl -->


<!-- Date: Sat Feb 7 18:34:03.038 2004 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178882">178882</A>
ACTIVATE: PHYSICS_2_01 [4.413,424] with reset central cal HV, TOF HV inhibit is masked
<!-- Author: Vadim x2080 -->



<!-- Date: Sat Feb 7 18:35:30.871 2004 -->
Also masked TOF00 inhibit
<!-- Author: Vadim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178882' target=_top>178882</a>) -->


<!-- Date: Sat Feb 7 18:43:55.408 2004 -->
Fotis and Nickolai came in and informed us that three channels of the central calorimeter had bad high voltages - off by 100 volts.  No alarm was issued by this condition, and only a subtle color variation in the CEM occupancy plot hinted at the problem (which we only saw in hind sight).  The run was ended, and we must mark it "bad" for CEM/CHA/WHA.  Apparently, they believe the high voltage system was corrupted last night when the PC was reset.  It was described as a "never before seen" event.
<!-- Author: Lukens :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881' target=_top>178881</a>) -->
<!-- Comment: Sat Feb  7 19;10;30 comment by...Nikolay and Fotis -->
At the end of yesterday's evening shift the pisabox pc was rebooted
because of problems with the heartbeat. At that moment the readout program was on a state of reading the system. This somehow put the system in a funny state for the first pmt of the readout for each subsystem. We observed that the HV for CEM 0W-0R, CHA 0W-0R and WHA 0W-6R were at 228, 216 and 163V lower than the nominal values (probably at the lower reachable limit of each pisa box but we are no sure for the actual value). Checking with YMON we saw that the occypancy of these pmts (at least for CEM and CHA) were lower than their partners. We reset the nominal values and instruct the shift crew to start another run. Since an alarm is set if there are more than 10 channels out of a 5V tolerance the only indication for something being wrong was the  asymmetric bars on the HV status display. We'll implement asap an alarm to prevent such kind of incidents. 
<!-- Comment: Mon Feb  9 09;57;40 comment by...Camille Ginsburg -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>Larry Nodulman, CAL SPL, says these data are 
nevertheless to be marked GOOD.</pre></td></tr></td></table>
<!-- Comment: Mon Feb  9 19;18;51 comment by...Fotis -->
As of Monday (02/09/04) there will an alarm if any single pmt drifts more than 30V out of range. See <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2004&action=view&page=119&frame=2&anchor=13:06:18.665'>
here </a> for a complete entry
<!-- Date: Sat Feb 7 19:03:47.868 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2950&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2950' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2951&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2951' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2952&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2952' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2953&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2953' width='300'></a></center>
<!-- Author: natasha -->



<!-- Date: Sat Feb 7 19:20:54.701 2004 -->
Gervasio made an extensive entry in the TOF e-log 
<a href= 
'/cgi/elog/det-system-elog.pl?nb=tof&action=view&page=15&frame=2&anchor=18:51:07'>here</a>, but to summarize:</p> 
 
<ul> 
<li>The two spares Matt and Gervasio tried (the TOF spare and then the plug spare) 
did not work because they had not set the crate ID to 1.  Once we did this with the 
plug crate (with some difficulty as explained below), it worked correctly.</li> 
<li>Unfortunately, we found the channels in the the card in slot 0 could not be 
turned on in this crate.  We tried two different cards with the same result, so the  
mainframe slot is the suspect.</li> 
<li>Also, this plug spare had others problems: using the front panel buttons to 
debug, we found that if any button was pressed in more than momentarily, the  
CAEN immediately froze.  Sometimes just touching or tapping on the front of 
the CAEN resulted in the same freeze.</li> 
<li>The TOF spare is very old with a completely different front panel.  Daniele also 
thinks it is running with very old firmware.  We need to work on the CAEN mainframe spare situation.</li> 
<li>Lastly, Gervasio also found one new channel (crate 2, slot 7, channel 1) which 
was not reading back current.  In the end, Gervasio thought it would be more  
confusing leaving HV on for the TOF with so many bad channels, and so currently 
the HV is off.</li>
<!-- Author: Steve Hahn -->
<!-- Comment: Sat Feb  7 23;32;46 comment by...HTML newb -->
Can the list be closed in a comment?
</ul>

<!-- Date: Sat Feb 7 20:32:53.345 2004 -->
<pre>L2 decision timeout. Did reboot_alpha. 
 
(MLE) b0l2de00:SpyAlpha:8:26:36 PM-&gt;ClistMon: Error on pass 0, cluster 0: 3 != 0 
(MLE) b0l2de00:SpyAlpha:8:26:41 PM-&gt;CListMon: Dumping CLIST data:Word       upper 32 bits  lower 32 bits 0: 0x8034b801	0x20020013  1: 0x0034b801	0x00020013  2: 0x0734b801	0x20020013  3: 0x0734b801	0x00020013 ClistMon: done. 
CT: 2004.02.07 20:26:25 
26'25" b0dap74.fnal.gov:resmgr.ReplyMonitor 4:8:26:26 PM-&gt; Dead Time is High  --&gt;  </pre>
<!-- Author: Vadim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178882' target=_top>178882</a>) -->
<!-- Comment: Sat Feb  7 23;30;17 comment by...Tom Wright -->
Please try HRR first on these before rebooting the Alpha, it probably would have recovered it.

<!-- Date: Sat Feb 7 20:45:36.133 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2954&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2954' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2955&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2955' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2956&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2956' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2957&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2957' width='300'></a></center>
<!-- Author: natasha -->


<!-- Date: Sat Feb 7 22:08:18.993 2004 -->
L2 decision timeout. Auto HRR cleared it. 
 
(MLE) b0l2de00:SpyAlpha:10:05:06 PM->Error: Startload is still low. ModDone bits 0x4007f7ff for event 84743843 
(MLE) b0l2de00:SpyAlpha:10:05:07 PM->TIMEOUT: Mod done not set for slot 14 (Clist)(0x0007f7ff, cnt 1) 
(MLE) b0l2de00:SpyAlpha:10:05:07 PM->ClistMon: No data from Clist! 
(MLE) b0l2de00:SpyAlpha:10:05:07 PM->ClistMon: Error on pass 0, cluster 0: 0 != 3 
CT: 2004.02.07 22:04:57 
4'57" b0dap74.fnal.gov:resmgr.ReplyMonitor 4:10:04:58 PM-> Dead Time is High [MLE] 
(MLE) b0l2de00:SpyAlpha:10:05:25 PM->Error: Startload is still low. ModDone bits 0x4007f7ff for event 84744164 
(MLE) b0l2de00:SpyAlpha:10:05:25 PM->TIMEOUT: Mod done not set for slot 14 (Clist)(0x0007f7ff, cnt 3) 
(MLE) b0dap73.fnal.gov:Thread-36816:10:05:02 PM->  Level 2 Decision Timeout
<!-- Author: Vadim :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178882' target=_top>178882</a>) -->


<!-- Date: Sat Feb 7 22:31:29.157 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2958&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2958' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2959&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2959' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2960&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2960' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2961&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2961' width='300'></a></center>
<!-- Author: natasha -->
<!-- Comment: Sat Feb  7 22;33;34 comment by...natasha -->
Note a delta function - like spike in antiproton abort gap losses!!! The scale plotted is 100kHz! Is the spike real? We had no alarms or even warnings from TevMon (do we alarm on antiprotons?)


<!-- Date: Sat Feb 7 22:55:55.813 2004 -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>A plan for rest of this shift and owl shift.... 
 
If we have to start a new run, please switch the  
trigger table to PHYSICS_2_01[4,414,424] (it contains 
additional debugging printouts for L2 problems).  
If anything appears wrong or unusual, switch back 
to the 413 table - otherwise use the 414 version for 
any physics running. 
 
If we have to start a new run anytime after midnight, please: 
 
1) Test TEST_ALPHA_CLUSTERING_NOSPIKES[2,409,403] 
   * Start with run configuration AAA_CURRENT 
   * Switch trigger table to  
       TEST_ALPHA_CLUSTERING_NOSPIKES[2,409,403] 
   * If anything appears grossly wrong, stop the test, 
     otherwise run for 20-25 minutes 
 
2) Do ZERO bias run for CLC 
   * Start with run configuration AAA_MINBIAS 
   * Under Triggers, select L1,2 Special Trigger Type 
     (decoupled from L3) Then select ZERO_BIAS_90HZ[2,412]  
      and [403]L3_PASS_ALL_RECO... 
   * Please take about 300k zero bias events. 
   * If any problems, return to regular physics running. 
 
If we have not started this work by 0500 on Sunday owl 
shift, please end physics run and do 1) and 2) and then 
return to physics running with the PHYSICS_2_01[4,414,424] 
table. 
 
If store is scheduled to run until at least noon, we  
will so Silicon Bias Scan for last 3 hours of store.</pre></td></tr></td></table>
<!-- Author: JJ -->
<!-- Comment: Sun Feb  8 07;18;44 comment by...Christopher -->
Tests 1 and 2 completed in owl shift. <br> Physics running resumed with PHYSICS_2_01[4,414,424]
<!-- Date: Sat Feb 7 22:57:21.658 2004 -->
It appears that Steve Hahn's entry about the TOF HV 
may have left an unclosed html tag. All entries since 
his entry are indented.
<!-- Author: jj -->



<!-- Date: Sat Feb 7 23:55:39.294 2004 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178881>
178881</A> 
x2BAC1</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_2_01 [4,413,424]</TD>                                       
<TD ALIGN=CENTER>02:46:58</TD>                                          
<TD ALIGN=CENTER>18:27:15</TD>                                          
<TD ALIGN=CENTER>14:48:48</TD>                                          
<TD ALIGN=RIGHT>    766,472,784</TD>                                    
<TD ALIGN=RIGHT>    14,455,794</TD>                                     
<TD ALIGN=RIGHT>   3,169,683</TD>                                       
<TD ALIGN=RIGHT> 1794.461</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=178882>
178882</A> 
x2BAC2</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_2_01 [4,413,424]</TD>                                       
<TD ALIGN=CENTER>18:31:40</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>05:12:05</TD>                                          
<TD ALIGN=RIGHT>    274,477,491</TD>                                    
<TD ALIGN=RIGHT>     3,821,450</TD>                                     
<TD ALIGN=RIGHT>     924,753</TD>                                       
<TD ALIGN=RIGHT>  374.933</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>23:55:02</TD>                                                       
<TD ALIGN=CENTER>20:00:54</TD>                                          
<TD ALIGN=RIGHT>   1,040,950,275</TD>                                   
<TD ALIGN=RIGHT>     18,277,244</TD>                                    
<TD ALIGN=RIGHT>     4,094,436</TD>                                     
<TD ALIGN=RIGHT> 2169.394</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Sat Feb 7 23:55:40.147 2004 -->
<font color=red>Shift Summary: </font><pre>Inherited run 178881 - running without TOF, which had a high voltage problem 
 
Central Calorimeter experts came in and informed us that the first three high voltage channels on each of CEM, CHA and WHA were way off.  They reset the voltages, and we stopped run 178881.  They advised setting the data "bad" for the calorimeter. I have decided to let more experts decide on this one, so the goodrun list has not been entered for this run. 
 
Lots of problems in Pbar and the booster this shift.  Stacking	began around 21:30.  Current plan has this store in until Sunday afternoon. 
 
Lum = 18E30 
Stack =	30 mA 
 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   178881,178882
Delivered Luminosity   602  
Acquired Luminosity    575  
Efficiency             0.96
   
</td></tr></table></pre>
<!-- Author: Lukens -->


<!-- Date: Sun Feb 8 00:15:14.096 2004 -->
<center><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2962&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2962' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2963&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2963' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2964&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2964' width='300'></a><a href='/cgi/elog/elog.pl?nb=2004&action=view&page=-2965&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2004&action=view&page=-2965' width='300'></a></center><font color=red><cite>(entry outside this shift's time range )</cite></font>
<!-- Author: Simon -->