<!-- Title: Day shift. Sun Jul 20, 2008 -->
<!-- SciCo: Ashutosh Kotwal -->
<!-- DAQAce: Mohammad Hussein -->
<!-- MonAce: &nbsp; -->
<!-- CO: Peter Bussey -->
<!-- OpManager: Peter Dong -->
<!-- Notes: -->
<pre>store 6306 colliding at 96E30</pre>


<!-- Date: Sun Jul 20 08:15:39.655 2008 -->
Peter Dong called to say talk to DAQ expert and then talk to Silicon expert to explore the possibility of putting silicon back in. 
<!-- Author: Ashutosh :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978' target=_top>264978</a>) -->

<!-- Date: Sun Jul 20 08:33:18.421 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
si: <tt>Silicon is not turned on this run</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 08:33:19.789 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
cal: <tt>PHA HV is yellow but this is said to be OK</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 08:33:20.095 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
smx: <tt>There is still a dead strip in CES wire canvas, since yesterday.<br>
We decided that this does not prevent the data from being good overall.</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 08:33:21.086 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
trigger: <tt>Silicon triggers inoperative of course.  Usual problems with IMU HAT</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->



<!-- Date: Sun Jul 20 08:36:31.144 2008 -->
DAQ expert Donatella says that from the DAQ perspective, it is probably OK to put the silicon back in and see what the trigger rate does. Suggests talking to Jonathan.
<!-- Author: Ashutosh :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978' target=_top>264978</a>) -->
<!-- Comment: Sun Jul 20 09;02;24 comment by...Ashutosh -->
I talked with silicon experts Mark and Satyajit, about putting silicon back in. Satyajit will call back with advice


<!-- Comment: Sun Jul 20 09;18;49 comment by...Ashutosh -->
Satyajit called to say lets try to put the silicon back in and see what happens. It is possible that L1 rate problem is not related to instantaneous luminosity. He will make an Elog entry. 
<!-- Comment: Sun Jul 20 09;19;54 comment by...Ashutosh -->
I informed Ops Manager Peter that we will put silicon back in and see what happens. He agrees. 
<!-- Comment: Sun Jul 20 09;20;46 comment by...Satyajit (Si SPL) -->
Now that the inst. lumi. is sufficiently low we can try putting 
silicon back in. If the problem appears we will take silicon out 
again until it is addressed. If the problem doesn't return in the 
remainder of the current store, we will include silicon at the 
beginning of the next store (under normal beam condition) as well.

<!-- Date: Sun Jul 20 09:02:42.651 2008 -->
CMX SW wedges 16-17 tripped. turn them on.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978' target=_top>264978</a>) -->


<!-- Date: Sun Jul 20 09:12:37.565 2008 -->
timeout fro crate cot_16. this is the 2nd time within 10 minutes. auto HRR fixed it.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978' target=_top>264978</a>) -->




<!-- Date: Sun Jul 20 09:21:38.057 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264975>264975</A>
smx: <tt>CES wire still has a missing section</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 09:21:39.365 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264975>264975</A>
general: <tt>No trigger plots to check.  Everything else normal.</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 09:25:39.291 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
 Terminated at 2008.07.20 09:25:11
<!-- Author: RunControl -->


<!-- Date: Sun Jul 20 09:25:40.510 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
TERMINATE: done
<!-- Author: Mohammad -->
<!-- Comment: Sun Jul 20 09;36;38 comment by...mohammad -->
Based on the request of SciCo I ended the no_silicon run and switch back to current one.
<!-- Date: Sun Jul 20 09:33:29.147 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979>264979</A>
 Activated at 2008.07.20 09:33:13
<!-- Author: RunControl -->


<!-- Date: Sun Jul 20 09:33:30.857 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979>264979</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_5_01 [7,1095,721] 
<!-- Author: Mohammad -->


<!-- Date: Sun Jul 20 09:36:38.426 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
SVX silicon removed due to large L1 trigger rate
ISL 
L00 
SVT 
</PRE>
<!-- Author: cdfscico -->



<!-- Date: Sun Jul 20 09:38:46.462 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264977>264977</A>
si: <tt>No silicon in this run</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 09:38:47.737 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264977>264977</A>
general: <tt>no silicon this run</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 09:38:48.242 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264977>264977</A>
RUNSTATUS: <PRE>Marked Bad, explanation:
SVX no silicon
ISL 
L00 
SVT 
</PRE>
<!-- Author: cdfscico -->



<!-- Date: Sun Jul 20 09:45:16.030 2008 -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>Dear shift crew,  
 
  Please take an ASDQ calibration when not taking data and  
BEFORE the access tomorrow morning . 
This is for saving reference data of electronics condition before 
the COT repair tomorrow.  
Please send an email to me when you are done. Many Thanks!!  
 
1. Choose "GenericCalibration" from the "Enable" pull-down menu.   
 
2. Pick your favorite partition number from the free partitions,    
 
3. Choose "Select Run Configuration" from the Parameters pull-down menu.   
 
4. Open the "Private" folder in the "Run Type Selector".   
 
5. Open the "Ambrose" folder.   
 
6. Choose "COT_ACE" (it should be highlighted) and click on "Select".   
 
7. Check if the RunType in the RunControlStatus is "COT ASDQ Test COT_ACE".   
 
8. Now you can go through the RunControl by clicking on "Partition",    
    "Config", "Activate" and "End".   
 
    If any crate misbehaves before "Activate", you may shepherd the crate.    
    If any crate misbehaves after "Activate", you should end the run and    
    shepherd the crate and restart RunControl.   
 
</pre></td></tr></td></table>
<!-- Author: Shin-Shan Yu (Eiko) -->
<!-- Comment: Sun Jul 20 09;45;54 comment by...eiko -->
If you dont get to do it in your shift, please pass this information to the evening shift and owl shift crew.
Thanks!!


<!-- Date: Sun Jul 20 09:57:29.839 2008 -->
I am following the L1 Trigger problem and for some runs I don't see the Trigmon slides (as for example run 264979, the current run). Is it any problem with TrigMon?  
Thanks
<!-- Author: Olga Norniella :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->
<!-- Comment: Sun Jul 20 11;02;26 comment by...Ashutosh -->
Maybe Trigmon slides are not generated for short runs?
<!-- Comment: Sun Jul 20 11;10;14 comment by...Heather -->
<table border=0><td><tr><td bgcolor=#e0e030>Please make sure TrigMon is running.  It has been "crashing", but not in a normal way.  It seems to give an error and just stop-- it is not understood.  If you see something like <a href=
'http://www-cdfonline.fnal.gov/cgi/internal/elog/elog.pl?nb=2008&action=copyentry&page=602&time=12:42:29.342' this </a>, please follow pasha's instructions and page consumer.  <br><br> It looks like TrigMon is not running now.  You must closely watch trigmon.  It seems this problem is correlated with the SL2 problems, but I couldn't say for sure.</td></tr></td></table>
<!-- Comment: Sun Jul 20 11;12;08 comment by...Heather -->
Apparently my link did not work.  I wanted to put in Peter's message from Sat Jul 19, 2008 at 12:42:29.  You can also check the status of the consumers from http://www-cdfonline.fnal.gov/daq/vxmon/conmgr/consumer_status.html.
<!-- Comment: Sun Jul 20 11;17;05 comment by...Heather -->
Another point of info.  When I have been running L2 tests, it seemed like TrigMon was not running at the beginning-- indicating the problem may have occurred at the end of the previous run (I am not 100% sure of this).  When this happened, CO on shift had to restart TrigMon.  I do not know if this happens every time, but I suggest always checking at the beginning of each run.
<!-- Comment: Sun Jul 20 11;18;11 comment by...Heather -->
at the beginning  -->  at the beginning of each run

<br> This observation was made on Thursday.
<!-- Comment: Sun Jul 20 13;08;10 comment by...Satyajit (evening CO) -->
Lately TrigMon crashes every now and then. Sometimes it gets 
through to save a root file and sometimes not. Pasha told us last 
evening that he would take a look at the crash details (which 
appears as a double mem delete in a d'tor).

<!-- Date: Sun Jul 20 10:02:37.502 2008 -->
We put the silicon back in, L1 accept rate seems to be holding steady
<!-- Author: Ashutosh :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->

<!-- Date: Sun Jul 20 10:05:04.503 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
smx: <tt>Still a bad set in CES wires</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 10:05:05.382 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264978>264978</A>
general: <tt>No silicon. &nbsp;Otherwise all faults are the normal ones.</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->



<!-- Date: Sun Jul 20 10:16:33.732 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-3688&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-3689  width='300' align='top'></a></td><td>silicon pinkie S_AVDD_B0W3L3, does not disappear with HRR</td></tr></table>
<!-- Author: Ashutosh -->
<!-- Comment: Sun Jul 20 10;32;17 comment by...Mark -->
If the powercycling that the Ace just did does not work, mark this cell as ignored, and I will hockerize the crate at EOS. 


<!-- Comment: Sun Jul 20 10;34;31 comment by...mohammad -->
The status is Halted waiting to hear from Sara-Si Expert.
<!-- Comment: Sun Jul 20 13;10;10 comment by...Satyajit -->
You probably meant SAR (the silicon auto-recovery program). After
every power cycle SAR kicks in to recover the switch off ladders.

<!-- Date: Sun Jul 20 10:32:48.810 2008 -->
I tried to run makeManyPlots (Xmon) for run 264978 
but after a number of cryptic messages it got stuck. 
(Yes, I allowed plenty of time...) 
This was a no-silicon run.  It seems to me that these programs 
should have a guaranteed terminating mechanism.
<!-- Author: Peter :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->








<!-- Date: Sun Jul 20 11:32:24.931 2008 -->
VRB_SVX_02 failed. HRR twice fixed it.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->

<!-- Date: Sun Jul 20 11:41:02.980 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979>264979</A>
general: <tt>OK apart from the normal problems. &nbsp;Trig mon had not had time to run yet.</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->



<!-- Date: Sun Jul 20 11:46:47.646 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-3690&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-3691  width='300' align='top'></a></td><td>silicon pinkie S_2VDOIM_B3W8L5 went away with HRR</td></tr></table>
<!-- Author: Ashutosh -->


<!-- Date: Sun Jul 20 11:49:00.021 2008 -->
ccal_07 failed, shepherd it.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->


<!-- Date: Sun Jul 20 11:50:40.792 2008 -->
timeout detected in cot_07. HRR fixed it.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->


<!-- Date: Sun Jul 20 11:55:21.768 2008 -->
ICICLE HB little box turns purple in the global alarm. Restarting ICICLE.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->
<!-- Comment: Sun Jul 20 12;05;27 comment by...mohammad -->
HB box is green now.


<!-- Date: Sun Jul 20 12:19:17.640 2008 -->
The sequencer is not running well. walking through the instruction starts the sequencer up and eventually return to the abort window. 
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->


<!-- Date: Sun Jul 20 12:19:32.570 2008 -->
I called CR to have them page consumers regarding Trigmon
<!-- Author: hkg -->
<!-- Comment: Sun Jul 20 12;22;57 comment by...Olga Norniella -->
I called too. They said they are looking on it.
<!-- Date: Sun Jul 20 12:22:37.793 2008 -->
shepherding cot17 failed twice. trying to shepherd it manually but its not responsive. calling expert to see if i need to power cycle it.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->
<!-- Comment: Sun Jul 20 12;26;14 comment by...mohammad -->
Danatella promptly called and she is checking the crate. she will call back soon.
<!-- Comment: Sun Jul 20 12;33;40 comment by...mohammad -->
Cryo will power cycle it and called us back.
<!-- Comment: Sun Jul 20 12;40;06 comment by...mohammad -->
cot17 is fine. back to run.
<!-- Date: Sun Jul 20 12:23:56.799 2008 -->
We have  TrigMon hanging in a similar way to yesterday. 
Page the responsible person and tell them about it.
<!-- Author: Peter :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->




<!-- Date: Sun Jul 20 12:37:35.580 2008 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2008&action=view&page=-3692&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2008&action=view&page=-3693  width='300' align='top'></a></td><td>S_AVDD_B0W3L2 pinkie does not go away with HRR</td></tr></table>
<!-- Author: Ashutosh -->
<!-- Comment: Sun Jul 20 12;58;32 comment by...Mark -->
You can mark this one as ignored too; it's in the same wedge as the previous pinky, which I will try to fix at EOS.
<!-- Comment: Sun Jul 20 13;29;33 comment by...Mark -->
I talked the SciCo thru hockerizing the PS crate containing this wedge. 
<!-- Date: Sun Jul 20 12:53:43.412 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264810>264810</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=264810 target=_runSummary>264810</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->



<!-- Date: Sun Jul 20 12:55:35.605 2008 -->
PSH and PHA alarm- crate 9 flashes red and sector 11 flashes yellow. I killed and reload the plug HV monitor and its running well but the warning is still there. paging PHA expert.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->


<!-- Date: Sun Jul 20 12:56:56.604 2008 -->
Following guidance from Pasha I have renamed the last TrigMon.cdfdaq.log file as /cdf/onln/data/cdfdaq/consumer/runarea/TrigMon/TrigMon_264979_log 
Now try to restart the TrigMon again
<!-- Author: Peter :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->



<!-- Date: Sun Jul 20 13:03:19.294 2008 -->
In the meantime of having PHA alarm, another alarm came from HBM. there is a hidden weird error behind the iFIX large window inj both vnode1 and vnode2 machines. click ok on both let the red color go away. 
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->
<!-- Comment: Sun Jul 20 15;18;40 comment by...mohammad -->
again
<!-- Date: Sun Jul 20 13:04:50.942 2008 -->
CMX NE wedge 21 tripped. turn it on. 
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->
<!-- Comment: Sun Jul 20 13;06;41 comment by...mohammad -->
Willis called and asked me to turn PSH,PHA off and then trun them on back in the meantime while I am fixing the CMX tripped wedge. this may fix the problem.




<!-- Comment: Sun Jul 20 13;56;55 comment by...mohammad -->
In the meantime of the first recycling for crate 9 crates 10 and 11 failed respectively. recycling again bringing back crates 9 and 10. recycling one more time fixed crate 11. thanks Willis.
<!-- Comment: Sun Jul 20 14;09;06 comment by...mohammad -->
same wedge for CMX. 
<!-- Comment: Sun Jul 20 14;15;22 comment by...mohammad -->
same wedge again. this time pull the bars up.
<!-- Date: Sun Jul 20 13:37:10.754 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979>264979</A>
general: <tt>Some normal problems remain. &nbsp;Keep an eye on TrigMon in next run.</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->



<!-- Date: Sun Jul 20 13:38:52.706 2008 -->
<center><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3694&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3694' width='300'></a><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3697&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3697' width='300'></a><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3698&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3698' width='300'></a><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3695&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3695' width='300'></a></center>
<!-- Author: jdl -->
<!-- Comment: Sun Jul 20 13;47;45 comment by...jdl -->
Donatella observed that the hand-brake problems of the last evening were tied to the phi-gap muon trigger.  Above are plots of the prefred rates for various triggers in various runs.
<bl>
<li>Top left: Run 264973, phi gap trigger.  The rate starts to get wonky near 23:20 and then flies off to the crossing rate.
<li>Top middle: same run, 15 GeV single track trigger.  Similar strange behavior near the end of the run
<li>Top right: Run 264976, phi-gap trigger.  Here the rate is ~300kHz.  What's up with that.  That fact totally messed up the L1 DPS for that run, too.  Perhaps trigmon will show something for this run.  Also, in this run, the L1_TRK15 trigger was fine, so it looks like a muon trigger issue.
<li>Bottom: Normal behavior of phi-gap trigger vs lumi taken from run 264953 in the previous store.
</bl>
This is unlike behavior I've seen in the past.  The effect in the tracks in the one run would point to XTRP.  But that doesn't hold with the later runs where there was trouble.  Maybe it is a muon trigger thing.  We've never had trouble with the muon trigger.  Eric is leaving on vacation in a few hours.  I don't think we have any phi-gap trigger plots in Trigmon, but there may be a nugget.


<!-- Comment: Sun Jul 20 14;31;15 comment by...Satyajit -->
Thanks, JDL/Donatella, for noticing this! Do we have enough
info. to conclude if this would reappear in the upcoming store?
If so, it would help us in deciding whether to include silicon
in future runs before this issue is resolved.

<!-- Date: Sun Jul 20 13:48:24.887 2008 -->
XMon complains that L2_MET_35 has a somewhat high trigger rate 
during runs 264978-9.  It lists some triggers that gave no 
output, namely L1_TWO_TRK2_B_HAD_NO_OPPQ_LUMI_65_v1 , 
B_CHARM_L1_LUMI_80_v5 , and B_PIPI_L1_LUMI_80_v5, 
but the latter two turned on partway through the run.  
<!-- Author: Peter :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->



<!-- Date: Sun Jul 20 14:00:18.555 2008 -->
<center><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3699&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3699' width='300'></a><a href='/cgi/elog/elog.pl?nb=2008&action=view&page=-3700&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2008&action=view&page=-3700' width='300'></a></center>
<!-- Author: jdl -->
<!-- Comment: Sun Jul 20 14;05;03 comment by...jdl -->
Trigmon does have info on XTRP.  So, I can cast blame in that direction.  Bits 13 and 14 are the phi-gap bits.  On the left is run 264976, and on the right is the reference 264953.  Note that in the bad run, several phi-gap track bits are on for each of the 14k events that Trigmon processed for that run.  Watch this canvas for trouble.  I'll send e-mail to Olga. 



<!-- Comment: Sun Jul 20 14;16;07 comment by...Olga Norniella -->
Yes, It looks like we have a problem in XTRP...in my way to FNAL to change that Data Board.


<!-- Comment: Sun Jul 20 14;22;14 comment by...jdl -->
It looks OK in the current run.  Maybe reloading it at coldstart was sufficient to fix it.
<!-- Date: Sun Jul 20 14:20:14.861 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264953>264953</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=264953 target=_runSummary>264953</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->





<!-- Date: Sun Jul 20 14:39:17.336 2008 -->
CMX NE wedges 18-19 tripped. pull the bars up.
<!-- Author: mohammad :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->


<!-- Date: Sun Jul 20 14:49:39.822 2008 -->
MCR called to say, store will be dropped in 15 mins
<!-- Author: Ashutosh :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979' target=_top>264979</a>) -->

<!-- Date: Sun Jul 20 14:52:35.415 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979>264979</A>
 Terminated at 2008.07.20 14:52:05
<!-- Author: RunControl -->



<!-- Date: Sun Jul 20 15:03:57.369 2008 -->
no quiet time given by MCR
<!-- Author: Ashutosh -->
<!-- Comment: Sun Jul 20 15;20;44 comment by...mohammad -->
this means I can not do CEM LED and CEM XEF non quiet time calibrations as both depend on CAL QIE quiet time calibration.
<!-- Comment: Sun Jul 20 15;46;28 comment by...Mark -->
Even though MCR may not give us quiet time, there is always about 15 minutes between stores where there is no beam...this is enough to do the three quick quiet-time calibrations :)
<!-- Date: Sun Jul 20 15:07:10.115 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264980>264980</A>
CALIBRATION: <U><B>tofqie</B></U> calibration <A HREF=http://www-cdfonline.fnal.gov/internal/daq/Calibrations/web.html#tofqie target=_calibresults>results</A> ready for review
<!-- Author: CDFCALIB -->


<!-- Date: Sun Jul 20 15:11:19.174 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264981>264981</A>
TERMINATE: done
<!-- Author: Mohammad -->
<!-- Comment: Sun Jul 20 17;25;53 comment by...Satyajit (CO) -->
No ACTIVATE entry for this run! Plots look fine. Calibration
successful.
<!-- Date: Sun Jul 20 15:13:38.081 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264982>264982</A>
ACTIVATE: RunSet: COTCALIB_ACE 
<!-- Author: Mohammad -->
<!-- Comment: Sun Jul 20 17;22;50 comment by...Satyajit (CO) -->
Calibration successful
<!-- Date: Sun Jul 20 15:13:39.498 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264982>264982</A>
TERMINATE: done
<!-- Author: Mohammad -->




<!-- Date: Sun Jul 20 15:22:08.276 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264979>264979</A>
general: <tt>Run appears to be OK apart from the usual minor items.</tt>
<!-- Author: Consumer Operator( Peter Bussey) -->


<!-- Date: Sun Jul 20 15:22:14.490 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264984>264984</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,1019,706] 
<!-- Author: Mohammad -->


<!-- Date: Sun Jul 20 15:25:36.021 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264812>264812</A>
<HTML>Run <A HREF=http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN=264812 target=_runSummary>264812</A> has not yet been checked for goodrun status; please do so.</HTML>
<!-- Author: RunControl -->


<!-- Date: Sun Jul 20 15:28:29.258 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264984>264984</A>
TERMINATE: end
<!-- Author: Mohammad -->


<!-- Date: Sun Jul 20 15:32:54.403 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264985>264985</A>
ACTIVATE: RunSet: ACE_COSMICS_ALL  Trigger: COSMICS [15,1019,706] 
<!-- Author: Mohammad -->
<!-- Comment: Sun Jul 20 15;56;52 comment by...Olga Norniella -->
I changed XTRP Data Board #11. There is a problem in this board that was effecting the phi-gap trigger. I checked with a cosmic run  that the DB is properly connected (as far as I can check with a cosmic run).  I will be around until the beginning of next store in case of problem.
<!-- Date: Sun Jul 20 15:50:41.515 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264985>264985</A>
 Terminated at 2008.07.20 15:50:13
<!-- Author: RunControl -->


<!-- Date: Sun Jul 20 15:53:20.010 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264986>264986</A>
TERMINATE: done
<!-- Author: Mohammad -->


<!-- Date: Sun Jul 20 15:55:27.489 2008 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>RunNumber</TH>
<TH>DataType</TH>
<TH>PhysicsTable</TH>
<TH>Begin</TH>
<TH>End</TH>
<TH>LiveTime</TH>
<TH>L3Accepts</TH>
<TH>DelivLumi nb<SUP>-1</SUP></TH>
<TH>LiveLumi nb<SUP>-1</SUP></TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=264978 target=_runSummary>264978</A> x40B12</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [7,1095,721]</FONT></TD>
<TD ALIGN=CENTER>07:01:12</TD>
<TD ALIGN=CENTER>09:25:11</TD>
<TD ALIGN=CENTER>02:17:32</TD>
<TD ALIGN=RIGHT>     565,404</TD>   <TD ALIGN=RIGHT>    825.68</TD>
<TD ALIGN=RIGHT>    783.53</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN=264979 target=_runSummary>264979</A> x40B13</TD>
<TD>BEAM</TD>
<TD><FONT SIZE=-1>PHYSICS_5_01 [7,1095,721]</FONT></TD>
<TD ALIGN=CENTER>09:33:13</TD>
<TD ALIGN=CENTER>14:52:05</TD>
<TD ALIGN=CENTER>03:29:40</TD>
<TD ALIGN=RIGHT>   1,201,031</TD>   <TD ALIGN=RIGHT>  1,437.44</TD>
<TD ALIGN=RIGHT>    957.12</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD>
<TD ALIGN=CENTER>1</TD></TR>

<TR><TD>Totals</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD>
<TD>15:55:01</TD>
<TD ALIGN=CENTER>05:47:12</TD>
<TD ALIGN=RIGHT>     1,766,435</TD> <TD ALIGN=RIGHT>  2,263.13</TD>
<TD ALIGN=RIGHT>  1,740.64</TD>
<TD> </TD>
<TD> </TD>
<TD> </TD></TR>

</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Sun Jul 20 15:55:38.266 2008 -->
TrigMon - crashes more times.  Olga has made the observation 
that TrigMon crashes _when_the_main_run_is_stopped!!! 
Then you must go to the window and press ctlC and wait for  
about ten minutes.  Then it can be restarted.
<!-- Author: Peter :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264987' target=_top>264987</a>) -->

<!-- Date: Sun Jul 20 15:56:16.934 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264987>264987</A>
ACTIVATE: RunSet: TOF_TAC_CALIB 
<!-- Author: Mohammad -->



<!-- Date: Sun Jul 20 16:00:36.287 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264987>264987</A>
TERMINATE: Done
<!-- Author: Mohammad -->



<!-- Date: Sun Jul 20 16:03:56.883 2008 -->
<font color=red>Shift Summary: </font><pre>Some pinkies, hockerization fixed them. Ran with silicon and L1 accept rate was OK. Some plug HV problems. XTRP board was changed. </pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   264978-9
Delivered Luminosity   1946.4/nb  
Acquired Luminosity    1411.5/nb  
Efficiency             72.5
   
</td></tr></table></pre>
<!-- Author: Ashutosh -->

<!-- Date: Sun Jul 20 16:05:01.289 2008 -->
Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=264988>264988</A>
ACTIVATE: RunSet: AAA_SHOTSETUP  Trigger: PHYSICS_5_01 [7,1095,721] 
<!-- Author: Mohammad -->