<!-- Title: Owl shift. Mon Oct  9, 2000 -->
<!-- Leader: Rick Tesarek -->
<!-- Ace: Dave Ambrose -->
<!-- Co1: Antoni Munar -->
<!-- Co2: Jonatan Piedra -->
<!-- Co3: David Saltzberg (SOS1) -->
<!-- Co4: Tom LeCompte (SOS2) -->
<!-- Co5: &nbsp; -->
<!-- Notes: -->
<pre>Continuing run 100837, default prescales, table 2. 
 
After this run, we will try new (special trigger table) 
L3_PASS_ALL_NULL.  If this works, will then take runs  
with #1 and #2 prescales, normal Myron (right now we're 
using L1 early), and then no Myron.</pre>


<!-- Date: Mon Oct 9 00:55:21 2000 -->
<pre>Some current rates for run 100837: 
 
--> from CDF Trigger Rate Monitor:  bit 0 = 1290 Hz 
                                        4 = 4900 
                                       12 = (same) 
                                       21 = 0 
                                       23 = 72 
--> from CDF Trigger Supervisor Rate: 30 Hz 
--> from Event Builder Daqmon: Partition 0 = 29 Hz</pre>
<!-- Author: dave :: (run 100837) -->


<!-- Date: Mon Oct 9 01:19:43 2000 -->
<pre>Ending run 100837, going to try special trigger table. 
coldstart took 200 attempts for L3. On activate, we get 
RXPT timeout after 1 event on most crates. 
 
Try again after rebooting tsi00.  It worked, now taking 
run 100840 with L3_PASS_ALL_NULL special trigger type, 
in Myron mode, L1 early.  Event builder rate about 200 Hz.</pre>
<!-- Author: dave :: (run 100840) -->


<!-- Date: Mon Oct 9 01:39:53 2000 -->
<pre>Ended run 100840 after 100k events (default prescales). 
Now going to switch to #1 prescale configuration (keeping 
the special trigger type L3_PASS_ALL_NULL).  Followed 
ace instructions to switch to scenario #1, and started  
run 100841.</pre>
<!-- Author: dave :: (run 100841) -->
<!-- Comment: Mon Oct  9 01;43;11 comment by...dave -->
<pre>some run 100841 stats:

--> event builder rate: 200 Hz
--> Trigger Supervisor Rate: 200 Hz
--> Trigger Rate Monitor: bit 0 = 1200 Hz
                              4 = 0
                             21 = 0
                             23 = 78

will take 200k.</pre>

<!-- Comment: Mon Oct  9 01;50;38 comment by...dave -->
<pre>currently have all CAL crates except:

--> PCAL04
--> WCAL02
--> WCAL07</pre>


<!-- Date: Mon Oct 9 02:14:18 2000 -->
<pre>Ended run 100841 (early Myron, trig scenario #1).  Now  
going to take normal Myron mode (uncheck "L1Early" and  
"IgnoreBC" boxes in run config).  By the way, we're using 
ACE_10-8 config file.  All above runs have subfarms 8-11. 
New run is 100842.</pre>
<!-- Author: dave :: (run 100842) -->


<!-- Date: Mon Oct 9 02:53:35 2000 -->
<pre>Ended run 100842 (normal Myron) at 210k. 
Now going to take "no Myron" run (uncheck Myronmode box), 
also add in PCAL04 (we can't find a reason why this has 
been excluded; WCAL crates removed have power supply problems). 
New run is 100843.  Get RXPT timeout after 5747 events, 
timeout on PCAL04.  Retry (run 100844) and get same error at 
exact event number 5747.  Spying on PCAL04 crate, got these 
messages: 
 
Monitor: interval 6000, state ACTIVE for partition 0 
Monitor: Events sent/readout/triggered 5085/5085/5086 
Monitor: Error status / VRB busy flag  0/0 
Monitor: readout/sent rate 203.720383 Hz / 203.720383 Hz 
TRACER_enableDone: Front End Fatal Error (-1): Tracer done not set 
No valid calibration for download 
Monitor: interval 6000, state ACTIVE for partition 0 
Monitor: Events sent/readout/triggered 5747/5747/5747 
Monitor: Error status / VRB busy flag  0/0 
Monitor: readout/sent rate 93.016724 Hz / 93.016724 Hz 
No new event ... 
Monitor: interval 6000, state ACTIVE for partition 0 
Monitor: Events sent/readout/triggered 5747/5747/5747 
Monitor: Error status / VRB busy flag  0/0 
Monitor: readout/sent rate 0.000000 Hz / 0.000000 Hz 
No new event ... 
Monitor: interval 6000, state ACTIVE for partition 0 
Monitor: Events sent/readout/triggered 5747/5747/5747 
Monitor: Error status / VRB busy flag  0/0 
Monitor: readout/sent rate 0.000000 Hz / 0.000000 Hz 
 
Removing PCAL04 and trying again, run 100845.</pre>
<!-- Author: dave :: (run 100845) -->


<!-- Date: Mon Oct 9 02:57:11 2000 -->
<pre>Sergei says CLC is ready to be added to next run.</pre>
<!-- Author: dave -->
<!-- Comment: Mon Oct  9 03;17;51 comment by...dave -->
<pre>Finished run 100845 (no Myron, trig scenario #1).
Added in CLC00 for run 100846, get timeout after first event,
with Merlin message:

   Host b0eb16: SCPU-P0-E-VrbReadFailed: 
   Error reading VRB in slot 10 for event 1.</pre>


<!-- Comment: Mon Oct  9 03;25;57 comment by...dave -->
<pre>Retry with CLC, for run 100847, this time it worked, but
we're getting the bunchcounter mismatch errors.</pre>
<!-- Comment: Mon Oct  9 04;37;53 comment by...dave -->
<pre>Ended this run (took almost 700k to look at consumers for
awhile).</pre>
<!-- Date: Mon Oct 9 03:20:55 2000 -->
<pre>"TROUBLE" from Flammable Gas alarms on CMEX NE top. 
Currently "-0.4%" LEL 
</pre>
<!-- Author: Saltzberg (SOS) -->



<!-- Date: Mon Oct 9 04:10:28 2000 -->
<pre>3:37 CMP Bottom tripped.  (Wire trip on section at 270 degrees) 
Since the crates are out of the run, set to standby.  </pre>
<!-- Author: Tom LeCompte -->



<!-- Date: Mon Oct 9 05:09:54 2000 -->
<pre>CMU N2 inerting went into alarm for a few seconds on the  
730' platform Ifix page.  SOS went and took a look at it  
and nothing is out of the ordinary.  Keeping an eye on it 
nevertheless.</pre>
<!-- Author: Tom LeCompte -->


<!-- Date: Mon Oct 9 05:40:22 2000 -->
<pre>Noticed that Trigger Rate Monitor bit 4 was back at 4000ish, 
so now not sure about previous scenario #1 runs...will redo. 
(Again, have CLC in now, but no PCAL04, WCAL02,07 or SVX.) 
 
--> run 100848: Myron (L1Early), trigger scenario #1 
--> run 100849: Myron (normal),  trigger scenario #1 
 
Actually seeing about 1.3 Hz in trigger bit 21.  Now try to 
add SVX: 
 
--> run 100850: Myron (L1Early), trigger scenario #1 
--> run 100852: Myron (normal),  trigger scenario #1 
                (this run took two tries to coldstart, unknown 
                 reason) 
 
Getting lots of CLC bunch counter errors (always these slots): 
 
     Bunch counter mismatch in event (n) 
     bunch counters in slot 3 and slot 19 disagree (at least) 
 
Suppressing these might be helpful (reducing by factor of 10).</pre>
<!-- Author: dave -->


<!-- Date: Mon Oct 9 06:00:31 2000 -->
<pre>Now taking trigger scenario #1 run with no Myron.  We have 
all CAL except PCAL04,WCAL02,WCAL07, we have the CLC, and  
we have SVX.  Run number 100853.  Event rate 190 Hz.  Trigger 
bit 0=1050 Hz, 21=1.1, 23=78, and all others zero.</pre>
<!-- Author: dave :: (run 100853) -->
<!-- Comment: Mon Oct  9 06;22;13 comment by...dave -->
<pre>This run has stopped taking data for several minutes, although
we have yet to get an error from run control.  Looking at
VxWorks Panel, we have CCAL09,11,12,15 with "corrupt" heaps.
Logging into CCAL09, get the following:

VxWorks login: vxworks
Password: 

b0ccal09-> Monitor: interval 6000, state ACTIVE for partition 0 
Monitor: Events sent/readout/triggered 23167/23167/23167
Monitor: Error status / VRB busy flag  0/0
Monitor: readout/sent rate 0.000000 Hz / 0.000000 Hz
0xe85cc0 (Messenger): Heap Error: 'Internal Inconsistency' at   e86a00 (count = 23228)
No new event ... 

Ended this run 100853, rebooting these CCAL crates (rebooted
WCAL02 while we're at it).</pre>

<!-- Comment: Mon Oct  9 06;37;51 comment by...dave -->
<pre>Tried run again, got immediate timeout on all crates after
1 event.
Try again, it runs, (number 100855).  This is "no Myron", with
trigger scenario #1, including most CAL, CLC, and SVX.</pre>


<!-- Comment: Mon Oct  9 07;06;45 comment by...dave -->
<pre>Run 100855 hangs again (with no errors).  Rebooted b0fib01,
which had no response from VxWorks.  Start same conditions,
run 100856.</pre>
<!-- Date: Mon Oct 9 06:53:02 2000 -->
<pre>IFix went to all ampersands (i.e. nothing is being read back for ECW and some solenoid monitoring.  Called cryo - they  
took down S2 for upgrades and forgot to tell us.  We won't be 
able to monitor anything on S2 for about an hour.</pre>
<!-- Author: Tom LeCompte -->



<!-- Date: Mon Oct 9 07:12:55 2000 -->
Changed the clock file from 36x36 to 1x8.
<!-- Author: Young-Kee -->
<!-- Comment: Mon Oct  9 14;14;16 comment by...SCN -->
This screwed up the SVX (see next entry).  Clock
Panel should look like <a href=/cgi/elog/elog.pl?nb=2000&action=view&page=154&anchor=14:12:26 target=_top>this</a>
<!-- Date: Mon Oct 9 07:25:27 2000 -->
<pre>Still trying to get non-Myron run going with scenario #1. 
Next try gives timeout, and b0fib00,2,4,6 go red on Ack display. 
Getting "statechange failed" errors on fibs.</pre>
<!-- Author: dave -->
<!-- Comment: Mon Oct  9 11;55;07 comment by...SCN -->
Clock Change Killed the Silicon.  Explanation 
 <a href=/cgi/elog/det-system-elog.pl?nb=barrel4&action=view&page=16&anchor=11:39:55 target=_top> here </a>
<!-- Date: Mon Oct 9 07:45:32 2000 -->
<pre>TrigMon and YMon remarks: 
(1) On B0dap24 TrigMon crashes. Console message: 
SysError in <TFile::WriteBuffer>: error writing to file L1Trigger.root (Stale NFS file handle) 
Everything stopped. PC reboot. 
(2) Display canvas must be modified. An entry with 
the number of events acquired should appear on the canvas of 
YMON. Statistics pad should be removed. Scale numbers are not 
seen. We include plots of calorimeter Towers, trigger towers 
and PM for CHAD and CEM for hot channels.</pre>
<!-- Author: Toni Munar J. Piedra :: (run 100855) -->


<!-- Date: Mon Oct 9 07:49:54 2000 -->
<pre>Finally got a run going again.  Ended up rebooting ccal1-6 and 
the tsi00.  Removed all SVX and CLC in the process of debugging. 
After clock change to 1x8, we now get: 
 
--> event rate: 8 Hz 
--> Trigger rate monitor: bit 0=35 Hz 
                              4=0 
                             21=0 
                             23=2.1</pre>
<!-- Author: dave :: (run 100862) -->
<!-- Comment: Mon Oct  9 08;15;18 comment by...dave -->
<pre>Got error because RF clock crate tripped.  On trying new run,
we're back to the fib0,2,4,6 hangs on coldstart.  Removed SVX
from run (placed power supply on standby).  Finally going
again (run 100864), back in Myron (L1Early), taking data at
about 90 Hz.</pre>
<!-- Comment: Mon Oct  9 08;32;38 comment by...dave -->
<pre>above run (100864) is accidentally in the default scenario.
In trying to restart, CCAL07 goes cold in coldstart (heap
error, internal inconsistency). rebooted.

Finally running again (100866), scenario #1, Myron(L1Early),
no SVX (but CLC!), rate is 30 Hz, bit0=35, bit23=2, rest 0.</pre>
<!-- Date: Mon Oct 9 07:51:05 2000 -->
<table><tr><td><a href=/elog/notebooks/2000/Mon_Oct__9_2000_Owl.075054.ps align=top><img src=/elog/gifs/ghost64.gif  width='56' align='top'></td><td>Trigger Tower</a></td></tr></table>
<!-- Author: Toni Munar J. Piedra -->


<!-- Date: Mon Oct 9 07:55:47 2000 -->
<table><tr><td><a href=/elog/notebooks/2000/Mon_Oct__9_2000_Owl.075243.ps align=top><img src=/elog/gifs/ghost64.gif  width='56' align='top'></td><td>CEM and HAD towers and PM for hot channels</a></td></tr></table>
<!-- Author: Toni Munar J. Piedra -->


<!-- Date: Mon Oct 9 07:56:26 2000 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2000&action=view&page=-293&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2000&action=view&page=-293  width='250' align='top'></td><td>Towers CEM</a></td></tr></table>
<!-- Author: Toni Munar -->


<!-- Date: Mon Oct 9 07:59:15 2000 -->
<table><tr><td><a href=/elog/notebooks/2000/Mon_Oct__9_2000_Owl.075901.ps align=top><img src=/elog/gifs/ghost64.gif  width='56' align='top'></td><td>CEM Towers</a></td></tr></table>
<!-- Author: Toni Munar -->


<!-- Date: Mon Oct 9 08:00:06 2000 -->
<table><tr><td><a href=/elog/notebooks/2000/Mon_Oct__9_2000_Owl.075951.ps align=top><img src=/elog/gifs/ghost64.gif  width='56' align='top'></td><td>HAD  Towers</a></td></tr></table><font color=red><cite>(Entry started on Mon Oct  9 08:00:01 2000)</cite></font>
<!-- Author: Toni Munar -->



<!-- Date: Mon Oct 9 08:20:18 2000 -->
<font color=red>Shift Summary: </font><pre>Had some success with the trigger scenario #1 masks. 
 
CLC; all CAL but PCAL04, WCAL02,07; no COT or SVX:  
 
--> run 100848: Myron (L1Early), trigger scenario #1  
--> run 100849: Myron (normal),  trigger scenario #1  
 
Add SVX:  
 
--> run 100850: Myron (L1Early), trigger scenario #1  
--> run 100852: Myron (normal),  trigger scenario #1  
 
Towards end of shift, are having bear of a time getting runs 
going.  Data taking will halt, but no errors are given by 
run control.  Have ended up removing the SVX, and (after 
trying to take non-Myron mode) are back to Myron-L1Early. 
 
</pre> 
<!-- Author: dave -->