<!-- Title: Eve shift. Fri Apr  1, 2005 -->
<!-- SciCo: Jaco Konigsberg -->
<!-- DAQAce: Xiaojian Zhang -->
<!-- MonAce: Xavier Portell -->
<!-- CO: teresa Rodrigo -->
<!-- OpManager: Aron Soha -->
<!-- Notes: -->
<pre>Will call MCR to arrange for Trigger Table test and flying wires 
and silicon Dmode calibrations</pre>

<!-- Date: Fri Apr 1 16:09:38.134 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196057">196057</A>
ACTIVATE: RunSet: TDCTEST_B0DAIG01 
<!-- Author: khotilov -->


<!-- Date: Fri Apr 1 16:17:58.381 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196058">196058</A>
ACTIVATE: RunSet: TDCTEST_B0DAIG01 till Sat noon
<!-- Author: khotilov -->



<!-- Date: Fri Apr 1 16:21:28.536 2005 -->
<pre>Got a PSM alarm in 1RR12D_5. Channel 3 was at 14V and it should be at 12V. The alarm went away quite soon. Now the value is 13.25 (still high) but seems fine.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196058' target=_top>196058</a>) -->


<!-- Date: Fri Apr 1 16:37:06.140 2005 -->
Called MCR a few minutes ago, no end-of-store plan yet.
<!-- Author: Jaco -->


<!-- Date: Fri Apr 1 16:39:50.463 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5406&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5406' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5407&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5407' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5408&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5408' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 16;40;09 comment by...Xavier -->
Smooth hourly plots.


<!-- Date: Fri Apr 1 17:13:47.336 2005 -->
L2 decision TO.
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196045' target=_top>196045</a>) -->
<!-- Comment: Fri Apr  1 17;29;45 comment by...Xiaojian -->
Another one. 


<!-- Date: Fri Apr 1 17:51:32.021 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5409&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5409' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5410&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5410' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5411&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5411' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5412&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5412' width='300'></a></center>
<!-- Author: Steve Hahn -->
<!-- Comment: Fri Apr  1 18;08;53 comment by...Steve Hahn -->
<h2>More clock commentary:</h2>

<p>Sorry this is so late, but I've been accumulating evidence all day, and discussing
it with Steve Chappa.  Aron Soha and Nils report that yesterday while they were
watching the crate, the error LEDs turned off with them touching the CLEAR ERROR
buttons.  This seems to agree with our hypothesis that what we saw last night about
2226 was consitent with the RESET bit being set in the PCC module.  To my 
knowledge, no one was using the clock GUI at this time; maybe a message to the
clock crate got garbled?</p>

<p>Also, there are some features of the beam pickups which may or may not be
significant.  The pictures above come from this scope.  The first is the proton 
pickup PW, which looks normal.  The second is the pbar pickup PbarE, which has
several features which look odd to me.  The largest signals are the proton bunches
as seen in the pbar pickup; the smallest signals are the pbar bunches.  However,
there is a third set of signals intermediate between these two which appears to
be delayed from the proton signals by about 1.12 us.  At first I thought these were
pbars in the wrong locations, but now I'm not sure what they are.  One possibility
Steve Chappa suggested are reflections but then we should we multiple attenuated
reflections and that doesn't seem to be the case.  The third plot shows the last
three of these signals which come after the last proton signal, and the fourth
plot shows these signals missing in the first three proton signals at the beginning
of the next group of 12 beam crossings.</p>

<p>One more Item: today about 1430 (?) Aron called me in since the BC LOC and 
the first two of the BC PHASE LEDs were on.  When we cleared them, they went 
away.  This would imply something was wrong in the PW pickup signal, but then it
went away.  Very odd.</p>
<!-- Date: Fri Apr 1 17:59:48.416 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196045">196045</A>
 Terminated at 2005.04.01 17:58:03
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 17:59:49.966 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196045">196045</A>
TERMINATE: end of run for silicon DMode calibration
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 17:59:49.018 2005 -->
Cryo reported problems with the EAST DRY ENGINE. Helium flow to solenoid is compromised. they may have to shut down the engine 
to switch to spare one (or the magnet will trip...). We are in 
a wait-and-see mode. 
 
I called MCR, so far they don't think they'll drop the store 
before midnight. 
 
Talked to ops manager, Aron, decided to start Silicon DMODE Calib now. 
 
stay tuned
<!-- Author: Jaco :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196045' target=_top>196045</a>) -->

<!-- Date: Fri Apr 1 18:06:02.858 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196059">196059</A>
ACTIVATE: RunSet: SVXCAL_DPSON  Trigger: COSMICS [14,503,540] , silicon dmode dspon
<!-- Author: Xiaojian x2080 -->


<!-- Date: Fri Apr 1 18:08:04.256 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196059">196059</A>
TERMINATE: end early due to bringing down silicon HV, solenoid is going to trip!
<!-- Author: Xiaojian x2080 -->




<!-- Date: Fri Apr 1 18:11:19.733 2005 -->
Cryo needs to bring down engine ASAP. We are turning HV down on 
relevant systems to avoid trips. more later  
<!-- Author: jaco -->


<!-- Date: Fri Apr 1 18:16:34.274 2005 -->
<pre>For the Solenoid problems, I put COT in off, Silicon to stdby and TOF to off. </pre>
<!-- Author: Xavier -->


<!-- Date: Fri Apr 1 18:23:37.158 2005 -->
MCR called to try to schedule our ~2hrs of stable beam for 
trigger tests and the ~1hr for flying wires. There are about 
 3 hrs till they drop the store. We want the last two hours 
because it'll take us about 1 hr to switch over to the spare dry  
engine.
<!-- Author: Jaco -->


<!-- Date: Fri Apr 1 18:27:48.442 2005 -->
After we ended the previous physics run for silicon DMode calibration. L3 has one node (c03) became orange.Reset from run control, the partition. It did not help. We have tried to clean L3, but it did not help too. Expert paged. 
<!-- Author: Xiaojian -->


<!-- Date: Fri Apr 1 18:29:26.043 2005 -->
Error message: "TevMon failing to receive Acnet messages via RTServer."
<!-- Author: Xiaojian -->
<!-- Comment: Fri Apr  1 19;53;46 comment by...Xiaojian -->
Another one. 
<!-- Date: Fri Apr 1 18:38:47.966 2005 -->
<pre>Bringing back again everything. Got permission!</pre>
<!-- Author: Xavier -->


<!-- Date: Fri Apr 1 18:39:25.246 2005 -->
MCR called. We have a "new" schedule: 
 
* The store will run till midnight 
* We'll have the last three hours to split between 
  flying wires and our trigger/calib tests 
* They will give us from 9 PM till 11 PM for our 
  tests and flying wires from 11 till midnite
<!-- Author: Jaco -->


<!-- Date: Fri Apr 1 18:41:15.240 2005 -->
Cryo called, they are actually ready, switched dry engines, no danger, we can power back up the COT/Sili/TOF.  
<!-- Author: Jaco -->


<!-- Date: Fri Apr 1 18:41:40.227 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5414&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5414' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5415&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5415' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5416&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5416' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 18;42;33 comment by...Xavier -->
Ups! I was sure I put last our plots... Now I put last two hours here! Sorry! It is all smooth.


<!-- Date: Fri Apr 1 18:44:14.390 2005 -->
Ok, given that we are back up and that everyone needed is in 
the control room anyway, we will start our tests NOW and get them 
over with.  
<!-- Author: Jaco -->

<!-- Date: Fri Apr 1 18:45:39.221 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196060">196060</A>
ACTIVATE: RunSet: SVXCAL_DPSON  Trigger: COSMICS [14,503,540] , silicon DMode dspon calibration.
<!-- Author: Xiaojian x2080 -->


<!-- Date: Fri Apr 1 18:50:57.388 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196060">196060</A>
TERMINATE: dpson ends.
<!-- Author: Xiaojian x2080 -->


<!-- Date: Fri Apr 1 18:54:35.531 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196061">196061</A>
ACTIVATE: RunSet: SVXCAL_DPSOFF  Trigger: COSMICS [14,503,540] , start silicon DMode dspoff
<!-- Author: Xiaojian x2080 -->


<!-- Date: Fri Apr 1 18:59:22.273 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196061">196061</A>
TERMINATE: RunSet: SVXCAL_DPSOFF  Trigger: COSMICS [14,503,540] , done silicon DMode dspoff calib
<!-- Author: Xiaojian x2080 -->


<!-- Date: Fri Apr 1 19:05:48.794 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196062">196062</A>
 Activated at 2005.04.01 19:05:01
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 19:06:15.300 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196062">196062</A>
ACTIVATE: RunSet: AAA_NOSILICON  Trigger: PHYSICS_TEST_3_02 [3,626,560] testing new table
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 19:13:56.890 2005 -->
<pre>L2 decision TO.  
 
(RC)   19:05:12 Activate -&gt; ACTIVE 
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:06:33 PM-&gt; 
L2receive: Too many CList clusters 
 
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:06:34 PM-&gt; 
L2receive: IsoList pass 1, cluster 1: got 0, expected 1 (eta: 12 phi: 22) 
 
 ... skip about 100 lines of the similar message. ... 
(MLE) b0dap73.fnal.gov:Thread-1111:Apr 1, 2005 7:05:27 PM-&gt;  Level 2 Decision Timeout 
(MLE) b0dap73.fnal.gov:Thread-1111:Apr 1, 2005 7:05:27 PM-&gt;Requested Halt-Recover-Run issued [errmon] 
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:06:44 PM-&gt; 
L2receive: IsoList pass 0, cluster 67: got 0, expected 1 (eta: 0 phi: 0) 
 
.. skip 10 lines of similar message at the top ... 
 
(MLE) b0l2de00:Messenger:Apr 1, 2005 7:05:28 PM-&gt;Alpha in slot 3 is no longer processing at spot 65 
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:06:45 PM-&gt; 
L2receive: IsoList pass 0, cluster 73: got 0, expected 1 (eta: 0 phi: 0) 
 
... more of the similar message ... 
 
 
 
</pre>
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196062' target=_top>196062</a>) -->
<!-- Comment: Fri Apr  1 19;26;43 comment by...Xiaojian -->
We have altogether five L2 decision TO since the start of run196062.
<!-- Date: Fri Apr 1 19:14:48.956 2005 -->
Silicon D-mode Calibrations has been done in run 196060, 196061. The required form has been filled and submitted. 
<!-- Author: Xiaojian -->


<!-- Date: Fri Apr 1 19:28:00.687 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196062">196062</A>
 Terminated at 2005.04.01 19:27:41
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 19:28:37.485 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196062">196062</A>
TERMINATE: end no silicon test of new table.
<!-- Author: Xiaojian x2080 -->
<!-- Comment: Fri Apr  1 19;56;52 comment by...Veronica -->
Just a few comments, the only difference between this table and the default one is that we have redefined the clustering by changing the forward SEED to 127. While runnning we have observed many L2 Decision TO and one TS_LEVEL2_ERROR message.
Also, and I have observed this early on the morning, although run summary shows all the LUMI enable triggers kicking in, the ScalerMon only shows two of them. Bill restarted ScalerMon on the morning and we got them to appear, but it happened again in this run.

<!-- Date: Fri Apr 1 19:29:37.513 2005 -->
<pre>A new Error: " L2 error, Trigger Supervisor error! ", HRR fixed it. some more detail: 
  
 
(RC)   19:25:43 Run -&gt; ACTIVE 
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:27:58 PM-&gt; 
L2receive: Too many CList clusters 
 
(MLE) b0dap84.fnal.gov:ConsumerErrorRe:Apr 1, 2005 7:26:48 PM-&gt;DAQErrMon:DataSyncError: Error 74 Event 73639 Run 196062 TMPD TL2D ReadoutList mismatch GlobalReadoutList 0 BC 255  
(MLE) b0dap84.fnal.gov:ConsumerErrorRe:Apr 1, 2005 7:26:48 PM-&gt;DAQErrMon:DataSyncError: Error 75 Event 73639 Run 196062 TMPD TL2D L2Buffer mismatch GlobalL2Buffer 0 L2Buffer 255  
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:28:16 PM-&gt; 
L2receive: CList pass 0, cluster 2: 2 != 0 
 
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:28:16 PM-&gt; 
L2receive: CList pass 2, cluster 0: 2 != 0 
 
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:28:16 PM-&gt; 
Bad data: resetting FIFOs 
 
(MLE) b0dap84.fnal.gov:ConsumerErrorRe:Apr 1, 2005 7:27:04 PM-&gt;DAQErrMon:DataSyncError: Error 76 Event 75265 Run 196062 TMPD TL2D ReadoutList mismatch GlobalReadoutList 0 BC 255  
(MLE) b0dap84.fnal.gov:ConsumerErrorRe:Apr 1, 2005 7:27:04 PM-&gt;D</pre>
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196062' target=_top>196062</a>) -->

<!-- Date: Fri Apr 1 19:33:49.507 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063">196063</A>
 Activated at 2005.04.01 19:33:35
<!-- Author: RunControl -->



<!-- Date: Fri Apr 1 19:34:50.950 2005 -->
WCAL_03 done TO.
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063' target=_top>196063</a>) -->

<!-- Date: Fri Apr 1 19:36:58.577 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063">196063</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_TEST_3_02 [3,626,560] testing table with silicon
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 19:41:28.520 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5417&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5417' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5418&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5418' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5419&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5419' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 19;41;54 comment by...Xavier -->
Hourly plots. All fine.


<!-- Date: Fri Apr 1 19:50:40.203 2005 -->
L2 decision TO, followed by "level 2 error: Trigger Supervisor erro!"
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063' target=_top>196063</a>) -->
<!-- Comment: Fri Apr  1 19;54;12 comment by...Xiaojian -->
Another L2 decision TO. 


<!-- Comment: Fri Apr  1 20;02;47 comment by...Xiaojian -->
Another one. 

<!-- Comment: Fri Apr  1 20;06;32 comment by...Xiaojian -->
Another one. followed by "level 2 error: Trigger Supervisor erro!"

<!-- Date: Fri Apr 1 20:08:55.747 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063">196063</A>
 Terminated at 2005.04.01 20:08:08
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 20:08:56.706 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063">196063</A>
TERMINATE: end test of trigger table with silicon
<!-- Author: Xiaojian x2080 -->
<!-- Comment: Fri Apr  1 20;18;41 comment by...Veronica -->
Well, more comments on this run, again we saw many L2 Decision TO and TS error. Also, this time the three LUMI enable triggers appeared on the ScalerMOn (I mean on the DynamicPrescale tab) but , although it looks fine on run summary, the last change , from 10 to 1 of L2_CEM12_ISO_DPS did not appear. 
We will look at the runs and then decide when to put this change  (DCAS change) on the default table. We want to do this at the same time we switch on the SVT long cluster cut.  
<!-- Comment: Fri Apr  1 22;39;04 comment by...Ted -->
<pre>Veronica: The error log for this run looks strange. In typical recent runs, alpha usually complains about cluster path from time to time and pull L2DTOs. But in this run, it looks like Alpha is a bit more unhappy about cluster path. in particular, it complains about "Too many CList clusters", and this is quite unusual. I don't remember when was last time i saw this type of error from alpha... with new DCAS threshold change, one would naively expect that there will be less clusters, not more. so the fact alpha is complaining about too many clusters is quite strange....don't know what exactly it really means. Maybe RunIIA alpha system experts can comment on this (how many is too many, and why? a hardware limitation?).
Note that Pulsar system is not complaining about cluser path for this run (note that TrigMon is feed by Pulsar TL2D, not Alpha TL2D now).

Anyway, it could be that alpha is unhappy about "too many cluster" for this run simply becuase it is in an unhappy mode on Friday night. but you are testing a new trigger table with DCAS cluster change, so maybe there is a good reason after all.

Ted

============== from error log ==========
(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:43:05 PM-&gt;
L2receive: CList pass 2, cluster 0: 1 != 0

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:44:31 PM-&gt;
L2receive: Too many CList clusters

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:44:31 PM-&gt;
L2receive: IsoList pass 1, cluster 3: got 1, expected 0 (eta: 19 phi: 22)

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:44:31 PM-&gt;
Bad data: resetting FIFOs

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:44:39 PM-&gt;
L2receive: Too many CList clusters

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:45:03 PM-&gt;
L2receive: Too many CList clusters

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:45:03 PM-&gt;
Bad data: resetting FIFOs

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:47:46 PM-&gt;
L2receive: Too many CList clusters

(MLE) b0l2de00:SpyAlpha:Apr 1, 2005 7:50:20 PM-&gt;
L2receive: Too many CList clusters
</pre>
<!-- Date: Fri Apr 1 20:10:54.084 2005 -->
End the run for trigger table test.  
 
b0cot14 became red, "Heap Corrupt Fatal Error!" 
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063' target=_top>196063</a>) -->
<!-- Comment: Fri Apr  1 20;11;47 comment by...Xiaojian -->
Shepherding fixed the bad crate. 


<!-- Date: Fri Apr 1 20:16:01.533 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064">196064</A>
 Activated at 2005.04.01 20:15:33
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 20:18:09.975 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064">196064</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_01 [2,618,559] , back to physics run.
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 20:18:28.670 2005 -->
L2 decision TO.
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064' target=_top>196064</a>) -->



<!-- Date: Fri Apr 1 20:38:44.382 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5420&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5420' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5421&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5421' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5422&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5422' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 20;39;09 comment by...Xavier -->
Smoot hourly plots.


<!-- Date: Fri Apr 1 20:48:20.523 2005 -->
MCR called. I informed them that we are done with our calibration/trigger studies and that in principle they 
ca do their flying wire when they want. They will call 
us in advance.
<!-- Author: Jaco -->


<!-- Date: Fri Apr 1 21:29:05.148 2005 -->
L2 decision TO.
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064' target=_top>196064</a>) -->


<!-- Date: Fri Apr 1 21:30:56.622 2005 -->
In DQMon, WHA became red. The error is  
 
"Calorimeter/WHAPMTCanvas Error Dead channel found in region [eta] different 6 or 7 No hot channels found. " 
 
It cleared itself after 1 minute. WHA is now green. 
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064' target=_top>196064</a>) -->


<!-- Date: Fri Apr 1 21:35:09.351 2005 -->
MCR called, will start flying wire at 9:45 PM, we'll bring 
stuff down 5 min before that and call then when ready
<!-- Author: Jaco :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064' target=_top>196064</a>) -->

<!-- Date: Fri Apr 1 21:40:59.947 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064">196064</A>
 Terminated at 2005.04.01 21:39:39
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 21:43:07.485 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064">196064</A>
TERMINATE: end of run for flying wire in store.
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 21:44:01.143 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5423&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5423' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5424&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5424' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5425&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5425' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 21;44;29 comment by...Xavier -->
Smooth hourly plots


<!-- Date: Fri Apr 1 21:48:12.917 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-5426&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-5427  width='300' align='top'></a></td><td>A pinky cell. It seems it has been doing strange things. The last drop is when we stopped the run for the flying wires.</td></tr></table>
<!-- Author: Xavier -->

<!-- Date: Fri Apr 1 21:57:21.700 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196065">196065</A>
TERMINATE: ihbvbkjkjn
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 21:58:56.361 2005 -->
<pre>We put things ready for the flying wires but it seems that we need to be on (except for the silicon) to test if something trips during the flying wires. When ramping up, CMP bottom A2 tripped twice. We put this part off. </pre>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 22;11;46 comment by...Xiaojian -->
Some more information: We have tried to ramp it up again but failed immediately. Since this section has had some bad trips for several cases in the last three month, see links below, and the only solution is to wait a number of hours. I decided to turn it off tight now (good thing is Lum = 16.5), and marked the run CMP bad. The store will end around midnight, I will then try to ramp it. The run should be marked as CMP bad.  

CMP bottom A2 trips, <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=81&frame=2&anchor=20:23:59.958'>
Jan 27th </a>, <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=186&frame=2&anchor=20:50:01.253'>
March 3rd </a>.
<!-- Date: Fri Apr 1 21:59:28.790 2005 -->
"Heap Corrupt Fatal Error! Please Shepherd crate b0xtrp00"  
 
Shepherding the crate fixed the problem.
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066' target=_top>196066</a>) -->

<!-- Date: Fri Apr 1 22:01:23.002 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066">196066</A>
 Activated at 2005.04.01 22:00:29
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 22:01:25.525 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066">196066</A>
ACTIVATE: RunSet: AAA_NOSILICON  Trigger: PHYSICS_3_01 [2,618,559] run with flying wire test.
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 22:03:20.667 2005 -->
talked to Ron, he will start the flying wire studies just about now, for about 1 hr. cross-checked with Aron that all should be on except silicon. Started Run 196066 unders these conditions.
<!-- Author: Jaco :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066' target=_top>196066</a>) -->



<!-- Date: Fri Apr 1 22:16:19.868 2005 -->
While we are runnning, each following audio message stops the run. 
 
"Partition 0, flying wire scan"  
 
Recover and Run resume the run. Jaco said the main control room is actually faking the flying wire signal now, not really flying the wire.  
  
<!-- Author: Xiaojian :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066' target=_top>196066</a>) -->


<!-- Date: Fri Apr 1 22:32:00.860 2005 -->
<table border=0><td><tr><td bgcolor=#e0e030>Run 196066 has flying wires: silicon is OFF, all other systems are ON. HOW DO WE KNOW WHICH EVENTS MAY HAVE BEEN TRIGGERED BY FLYING WIRES? an inhibit is supossed to be implemented, is it? is it timed 
properly?</td></tr></td></table>
<!-- Author: Jaco :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066' target=_top>196066</a>) -->
<!-- Comment: Fri Apr  1 22;39;50 comment by...Xavier -->
Just not to alarm anybody, say that the Silicon is in STDBY and not OFF. Jaco wanted to say that we are running the run without silicon.
<!-- Comment: Fri Apr  1 23;14;00 comment by...Ted -->
<pre>Jaco:  the info below may be useful...

--------------------------------------------------------------------------------
 Thu Mar 31 17:36:52 
Have upgraded the fer FrontEndReadout / Run Control  
package to the new version v3_68, with a couple minor  
changes: 

   . Comment out the special unique subject specification  
     for front-end crates (don't know how that keeps creeping  
     back in) just in case that causes rtserver problems. 

   . Automatically HALT the ongoing run if a flying wires  
     Acnet event message is received (code 0xCF) -- but  
     only if there is luminosity in the tevatron 

There you go, Willis! 

 - W.Badgett :: (run 196025)
-- Thu Mar 31 21:19:09 comment by...jdl --  The trigger inhibit is set up for this if you'd prefer. </pre>
<!-- Comment: Sat Apr  2 07;18;42 comment by...W.Badgett -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>But the "Flying Wires" inhibit signal is marked 
OFFLINE in the hardware database, so it will not 
be noticed by the system;   JDL would have to approve 
turning it back on.
</pre></td></tr></td></table>
<!-- Date: Fri Apr 1 22:37:20.758 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5428&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5428' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5429&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5429' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5430&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5430' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 22;38;44 comment by...Xavier -->
Hourly plots. You can see the flying wires. Silicon people don't worry! Silicon was in STDBY...



<!-- Date: Fri Apr 1 23:08:54.679 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066">196066</A>
 Terminated at 2005.04.01 23:08:34
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 23:09:27.938 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066">196066</A>
TERMINATE: end of run after flying wire test is over
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 23:09:54.078 2005 -->
<pre>Flying wires stopped. Things look smooth. I turn up SVX and also CMP (let's see if now it doesn't trip...)</pre>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 23;10;28 comment by...Xavier -->
No, it trips again. We turn this part off.


<!-- Date: Fri Apr 1 23:11:09.285 2005 -->
<pre>Got an ICICLE alarm some minutes ago but it looked fine so I ignored the message. It was cleared now.</pre>
<!-- Author: Xavier -->


<!-- Date: Fri Apr 1 23:15:01.853 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196067">196067</A>
 Activated at 2005.04.01 23:14:39
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 23:19:35.467 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196067">196067</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_01 [2,618,559] 
<!-- Author: Xiaojian x2080 -->



<!-- Date: Fri Apr 1 23:42:02.915 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5431&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5431' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5432&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5432' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-5433&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-5433' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Fri Apr  1 23;42;56 comment by...Xavier -->
Hourly plots. At the beginning there are very small peaks due to the flying wires but not really important. The rest is smooth...


<!-- Date: Fri Apr 1 23:48:12.114 2005 -->
MCR called, they'll go into shot setup at around 0:15 instead of 
 midnight. Some problems with the p-bar source.
<!-- Author: Jaco -->


<!-- Date: Fri Apr 1 23:49:25.094 2005 -->
Having problems saving the GOODRUN bits from the goodrun GUI, something related to privileges. Will try to sort it out tomorrow and fill out today's runs then.
<!-- Author: Jaco -->
<!-- Comment: Sat Apr  2 07;20;24 comment by...W.Badgett -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>Did you select "Change Database Connection" and press 
the "External" button?   (see instructions)</pre></td></tr></td></table>
<!-- Date: Fri Apr 1 23:55:42.678 2005 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196045>
196045</A> 
x2FDCD</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_01 [2,618,559]</TD>                                       
<TD ALIGN=CENTER>05:57:01</TD>                                          
<TD ALIGN=CENTER>17:58:03</TD>                                          
<TD ALIGN=CENTER>11:48:53</TD>                                          
<TD ALIGN=RIGHT>    742,831,911</TD>                                    
<TD ALIGN=RIGHT>     8,093,555</TD>                                     
<TD ALIGN=RIGHT>   2,240,368</TD>                                       
<TD ALIGN=RIGHT> 1024.127</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196062>
196062</A> 
x2FDDE</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_TEST_3_02 [3,626,560]</TD>                                  
<TD ALIGN=CENTER>19:05:01</TD>                                          
<TD ALIGN=CENTER>19:27:41</TD>                                          
<TD ALIGN=CENTER>00:20:21</TD>                                          
<TD ALIGN=RIGHT>      9,068,521</TD>                                    
<TD ALIGN=RIGHT>        79,849</TD>                                     
<TD ALIGN=RIGHT>      26,214</TD>                                       
<TD ALIGN=RIGHT>   21.712</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196063>
196063</A> 
x2FDDF</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_TEST_3_02 [3,626,560]</TD>                                  
<TD ALIGN=CENTER>19:33:35</TD>                                          
<TD ALIGN=CENTER>20:08:08</TD>                                          
<TD ALIGN=CENTER>00:30:30</TD>                                          
<TD ALIGN=RIGHT>     17,319,413</TD>                                    
<TD ALIGN=RIGHT>       203,298</TD>                                     
<TD ALIGN=RIGHT>      65,056</TD>                                       
<TD ALIGN=RIGHT>   31.978</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196064>
196064</A> 
x2FDE0</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_01 [2,618,559]</TD>                                       
<TD ALIGN=CENTER>20:15:33</TD>                                          
<TD ALIGN=CENTER>21:39:39</TD>                                          
<TD ALIGN=CENTER>01:23:10</TD>                                          
<TD ALIGN=RIGHT>     57,245,527</TD>                                    
<TD ALIGN=RIGHT>       646,823</TD>                                     
<TD ALIGN=RIGHT>     194,670</TD>                                       
<TD ALIGN=RIGHT>   84.739</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196066>
196066</A> 
x2FDE2</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_01 [2,618,559]</TD>                                       
<TD ALIGN=CENTER>22:00:29</TD>                                          
<TD ALIGN=CENTER>23:08:34</TD>                                          
<TD ALIGN=CENTER>00:47:12</TD>                                          
<TD ALIGN=RIGHT>     22,183,940</TD>                                    
<TD ALIGN=RIGHT>       184,224</TD>                                     
<TD ALIGN=RIGHT>      56,968</TD>                                       
<TD ALIGN=RIGHT>   45.505</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196067>
196067</A> 
x2FDE3</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_01 [2,618,559]</TD>                                       
<TD ALIGN=CENTER>23:14:39</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>00:38:09</TD>                                          
<TD ALIGN=RIGHT>     21,173,548</TD>                                    
<TD ALIGN=RIGHT>       244,115</TD>                                     
<TD ALIGN=RIGHT>      77,572</TD>                                       
<TD ALIGN=RIGHT>   36.943</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>23:55:04</TD>                                                       
<TD ALIGN=CENTER>15:28:17</TD>                                          
<TD ALIGN=RIGHT>     869,822,860</TD>                                   
<TD ALIGN=RIGHT>      9,451,864</TD>                                    
<TD ALIGN=RIGHT>     2,660,848</TD>                                     
<TD ALIGN=RIGHT> 1245.004</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->


<!-- Date: Fri Apr 1 23:59:19.038 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196067">196067</A>
 Terminated at 2005.04.01 23:58:19
<!-- Author: RunControl -->


<!-- Date: Fri Apr 1 23:59:20.287 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=196067">196067</A>
TERMINATE: end of store
<!-- Author: Xiaojian x2080 -->



<!-- Date: Sat Apr 2 00:01:26.207 2005 -->
<font color=red>Shift Summary: </font><pre>Got the calib and trigger tests done AND took some data too</pre> 
<!-- Author: Jaco -->