<!-- Title: Eve shift. Sat Jul 23, 2005 -->
<!-- SciCo: art garfinkel -->
<!-- DAQAce: svenja richter -->
<!-- MonAce: greg williams -->
<!-- CO: paola gianetti -->
<!-- OpManager: sungwon lee -->
<!-- Notes: -->
<pre>run 201607 in progress</pre>


<!-- Date: Sat Jul 23 16:14:37.164 2005 -->
Tevatron Flying Wires Scan
<!-- Author: Greg :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607' target=_top>201607</a>) -->
<!-- Comment: Sat Jul 23 17;20;25 comment by...Greg -->
again at 17:13:28
<!-- Comment: Sat Jul 23 18;13;36 comment by...Greg -->
again
<!-- Comment: Sat Jul 23 19;13;49 comment by...Greg -->
again
<!-- Date: Sat Jul 23 16:33:06.725 2005 -->
<pre>Jim Morgan's E-log shows that the store will end at 20:30. I called MCR and learned that there will be no quiet time but they will go  immediately into shot setup.</pre>
<!-- Author: garfinkel :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607' target=_top>201607</a>) -->


<!-- Date: Sat Jul 23 16:38:39.796 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12608&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12608' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12609&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12609' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12610&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12610' width='300'></a></center>
<!-- Author: Svenja -->
<!-- Comment: Sat Jul 23 16;39;03 comment by...Svenja -->
hourly plots 15:30-16:30


<!-- Date: Sat Jul 23 16:42:40.932 2005 -->
<pre>MCR just called to revise the schedule. Now they plan to terminate the store at 7:30 so that D0 can make a 1 hour access starting at 7:45. This means that we will have a bit more than an hour of quiet time. </pre>
<!-- Author: garfinkel -->
<!-- Comment: Sat Jul 23 17;17;21 comment by...Sungwon -->
<pre>informed that CDF will not go in. </pre>
<!-- Date: Sat Jul 23 16:43:42.808 2005 -->
<pre>3 consecutive DAQ errors: 
 
1st: SvxMon caused HRR to recover 527 "out-of-synch" chips 
2nd: Done Timeout on b0pcal09 -> Auto HRR worked to recover.   
3rd: Error Handler freezes 
At this point we were taking data again, but the Error Display froze and went blank.  The menu bar disappeared so I clicked the upper right "X" to close the window, and waited for about 30 seconds before it closed.  I then started a new error handler, and it seems to be working normally.   
Details from the error logger: 
CT: 2005.07.23 16:29:46 
29'46" b0dap84.fnal.gov:ConsumerErrorRe:Jul 23, 2005 4:29:46 PM->Runtime Error 3, Event 41039, RunNum 201607: SvxMon Halt Recover Run: Pipeline out of synch in 527 silicon readout chips . -->  Additional Information:  
 Attention !!!. CER_SVXMON_HALT_RECOVER_RUN_ERROR !!!  
  Pipeline out of synch in 527 silicon readout chips .  
 AUTO HRR will be issued  
[MLE] 
(MLW) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 4:29:46 PM->Received error message: errmon Active TurboHRR HRR suggested for:b0dap84.fnal.gov:ConsumerErrorRe:Jul 23, 2005 4:29:46 PM->Runtime Error 3, Event 41039, RunNum 201607: SvxMon Halt Recover Run: Pipeline out of synch in 527 silicon readout chips . 
(MLE) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 4:29:46 PM->Requested Halt-Recover-Run issued (errmon) 
(MLE) b0svx01:Messenger:Jul 23, 2005 4:29:56 PM->Silicon Resonator: Sl 12 Ch 7 -> e152: Bytes: 00140 
e140 18:e180 20:e1a0 
(RC)   16:29:50 Halt -> HALTED 
(RC)   16:29:57 Recover -> RECOVERED 
(RC)   16:30:00 Run -> ACTIVE 
(MLE) b0l3pcom1.fnal.gov:main:Jul 23, 2005 4:30:11 PM->Host b0eb16.fnal.gov, task tRec_0 
SCPU-P0-E-VrbReadFailed: Error reading VRB in slot 12 for event 18314984. 
The VRB had no event. 
CT: 2005.07.23 16:30:12 
30'12" 1 crate/s: b0pcal09(43),  Done TO. [RXPT] 
(MLE) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 4:30:12 PM->Done Timeout: PCAL_09  
(MLW) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 4:30:12 PM->Received error message: errmon Not a state HaltRecoverRun 1 crate/s: b0pcal09(43),  Done TO. [RXPT] 
(MLE) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 4:30:12 PM->Requested Halt-Recover-Run issued (errmon) 
(MLE) b0svx04:Messenger:Jul 23, 2005 4:30:10 PM->Silicon Resonator: Sl 10 Ch 7 -> e432: Bytes: 00136 
e440 16:e460 18:e480 20:e4a0 
 
(RC)   16:30:16 Halt -> HALTED 
(RC)   16:30:23 Recover -> RECOVERED 
(RC)   16:30:25 Run -> ACTIVE</pre>
<!-- Author: Greg :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607' target=_top>201607</a>) -->
<!-- Comment: Sat Jul 23 17;19;33 comment by...Greg -->
<pre>Donatella, the Error Handler expert, called after noticing that the error logger was not logging anything after I restarted it.  As it is described on the error handler webpage, what I should have done when I noticed the frozen display was to:
a) wait longer before doing anything
b) try issuing a PAUSE - RESUME to run control
c) if still no signs of life, close the error logger like I did, and then start it again and once it starts, click on the "errlog" client box and bring it individually through the Partition-Config-Activate steps.  That is what I missed doing, and why the error logger wasn't working after I restarted it.  After doing this it might be necessary to do a PAUSE - RESUME on Run Control to get it going again.  
d) if all else fails, it could be necessary to stop the run and do a RESET - Partition - Config - Activate for the whole system.</pre>
<!-- Comment: Sat Jul 23 17;59;14 comment by...Donatella T. -->
thanks for adding the comment. only one clatification : when the Error Handler is ACTIVE again, you MAY want to issue PAUSE-RESUME to quickly see that the EH Display is indeed updating OR wait for the next DAQ error message to check this.
About the 'freeze': in the session logfile there was a Java Swing Container error that I'm going to investigate further.
<!-- Date: Sat Jul 23 16:58:29.684 2005 -->
<pre>hi, Paola and the rest of shift crew: 
 
I have just added a few comments in the previous shift e-log, on some XMON plots and TrigMon plot posted by previous CO. (Paola also noticed similar problem plots yesterday as well).  Please take a look.  thanks.</pre>
<!-- Author: Ted  -->



<!-- Date: Sat Jul 23 17:16:28.467 2005 -->
The last flying wire was followed by a Roman Pots alarm which recovered within a second.
<!-- Author: Svenja :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607' target=_top>201607</a>) -->




<!-- Date: Sat Jul 23 17:29:04.162 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607">201607</a>
 Terminated at 2005.07.23 17:28:31
<!-- Author: RunControl -->
<!-- Comment: Sat Jul 23 18;25;27 comment by...Greg -->
<pre>So here we had some trouble with DAQ.  It started at ~5:21pm when the b0dap57 machine spontaneously reset and rebooted.  I was running DQMon, as usual in the upper screen and Mozilla in the lower screen.  I was scrolling down the e-log when it reset.
At 5:23 an auto-HRR was requested, but b0tsi00 got stuck going to the halt state.  I tried issuing the HALT transition manually to that individual crate, but was unsuccessful.  
Here's some error log details:
(MLW) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 5:23:54 PM->Received error message: errmon Not a state HaltRecoverRun 1 crate/s: b0svx06(16),  busy. [RXPT]
(MLE) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 5:23:54 PM->Requested Halt-Recover-Run issued (errmon)
CT: 2005.07.23 17:23:54
23'54" 1 crate/s: b0svx06(16),  busy. [RXPT]

Svenja came over to help, and we noted that this is not a shepherdable machine because it requires sequencing.  After trying to Halt it unsuccessfully, we issued an Abort command to the whole system.  Nothing happened.  Everything still was stuck in HRR sequence going to halt.  We did a manual override and again tried to abort, but it was still unsuccessful. We did observe that the Abort box stayed highlighted green after we pressed it. 
MLW) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 5:26:02 PM->StateManager: User has requested an over-ride!

We tried to Reset, since the Abort wasn't apparently working, but 
that too had no effect.  We did another manual override followed by a Reset.  That had an immediate effect, and many of the clients started changing color and resetting.  However, some of the clients were stuck, and Svenja tried aborting them each individually - and that worked.  

(MLW) b0dap73.fnal.gov:Thread-24:Jul 23, 2005 5:28:30 PM->StateManager: User has requested an over-ride!
(RC)   17:28:30 Recover -> RECOVERED
(RC)   17:28:32 Reset -> START

With everything in the start state, we tried to go through the partition-config-activate to start taking data again.  Many of the crates failed to config right, so we reset and started again.  

We paged the DAQ expert and Bill Badgett (I believe) responded right after we had restarted taking data.  He promised to look over the error logs and see if anything useful could be gleaned.</pre>
<!-- Date: Sat Jul 23 17:29:05.868 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607">201607</a>
TERMINATE: end run after b0tsi00
<!-- Author: Clark -->


<!-- Date: Sat Jul 23 17:34:41.427 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201624">201624</a>
TERMINATE: end run
<!-- Author: Clark -->
<!-- Comment: Sat Jul 23 18;27;00 comment by...Greg -->
I don't actually remember starting this run, but maybe after we pressed reset we should have waited longer.  
<!-- Date: Sat Jul 23 17:34:42.474 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625">201625</a>
 Activated at 2005.07.23 17:34:34
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 17:36:31.020 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625">201625</a>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_06 [1,683,585] Start run
<!-- Author: Clark -->



<!-- Date: Sat Jul 23 17:42:08.062 2005 -->
We got a procmon error because of FCDFSGI1. The reference on the web says that this doesn't require immediate action, but if this persists for more than 4 hours we should send an email to the experts. 
  
<!-- Author: Svenja :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625' target=_top>201625</a>) -->
<!-- Comment: Sat Jul 23 17;43;53 comment by...Sungwon and Art -->
paged CSL expert


<!-- Comment: Sat Jul 23 18;45;32 comment by...Greg -->
<pre>The CSL Expert, Jedong, replied to the page and we told him about the red flag about FCDFSGI1 TAPE; when we clicked on the red flag a ProcMon Actions window popped up and we ran a check:

Sat Jul 23 17:51:46 CDT 2005   procAction  fcdfsgi1   check   b0control01
Checking fcdfsgi1
check file modify time - Sat Jul 23 17:31:38 2005 <1122157898>  (should be recent) 
Found 6 entries in 'encp status' (0 expected)
fcdfsgi1 check failed

Jedong didn't know what this means, but assured us that the CSL system has enough disk space for 2-3 days of data taking, even with a high luminosity store coming in later tonight.  So if there is a problem with the tape writing, it doesn't prevent us from taking data.  Jedong advised me to email cdfdh_oper and describe the warning message we're getting.  I emailed them.</pre>
<!-- Comment: Sat Jul 23 23;22;00 comment by...Greg -->
Good news, the FCDFSGI1 error cleared.  
<!-- Date: Sat Jul 23 17:44:47.622 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12611&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12611' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12612&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12612' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12613&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12613' width='300'></a></center>
<!-- Author: Svenja -->
<!-- Comment: Sat Jul 23 17;45;08 comment by...Svenja -->
hourly plots 16:30-17:30



<!-- Date: Sat Jul 23 18:18:49.913 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607">201607</a>
monitors: <tt>Some of the XMON fits looks to have the wrong shape at high luminosity. See below plots and names.</tt>
<!-- Author: Consumer Operator(Paola) -->



<!-- Date: Sat Jul 23 18:22:27.018 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12616&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12616' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12617&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12617' width='300'></a></center>
<!-- Author: Paola -->
<!-- Comment: Sat Jul 23 18;27;47 comment by...Paola -->
Trigger xsec L2_CEM12_ISO_&_SUMET20_&_TWO_JET3_ETA1.8 is not
reproduced by fit at Low and High Lum. 


<!-- Date: Sat Jul 23 18:30:19.504 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12618&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12618' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12619&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12619' width='300'></a></center>
<!-- Author: Paola -->
<!-- Comment: Sat Jul 23 18;33;49 comment by...Paola -->
Trigger Xsec L2_CEM12_ISO_&_TRK2_D120_v3 is over the allowed
band at high Lum 


<!-- Date: Sat Jul 23 18:34:20.665 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12620&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12620' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12621&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12621' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12622&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12622' width='300'></a></center>
<!-- Author: Svenja -->
<!-- Comment: Sat Jul 23 18;34;39 comment by...Svenja -->
hourly plots 17:30-18:30


<!-- Date: Sat Jul 23 18:37:15.899 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12623&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12623' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12624&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12624' width='300'></a></center>
<!-- Author: Paola -->
<!-- Comment: Sat Jul 23 18;48;06 comment by...Paola -->
Trigger L2_JET90: the crosse section shape is not well reproduced
by the fit. At high Lum seems almost flat with Lum.


<!-- Comment: Sun Jul 24 09;13;16 comment by...convery -->
flat at high lum is good!
<!-- Comment: Sun Jul 24 16;22;59 comment by...Paola -->
Yes very good, but Xmon will be unhappy
<!-- Date: Sat Jul 23 19:21:45.280 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625">201625</a>
 Terminated at 2005.07.23 19:20:34
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 19:21:46.855 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625">201625</a>
TERMINATE: End of Store in 10 minutes
<!-- Author: Greg x2080 -->



<!-- Date: Sat Jul 23 19:24:43.158 2005 -->
<pre>Starting short EVB upgrade test. L3 and EVB disabled.</pre>
<!-- Author: markus -->


<!-- Date: Sat Jul 23 19:33:27.873 2005 -->
b0pcal04 was not responsive while we were in the start state. I shepherded it succesfully.
<!-- Author: Svenja -->


<!-- Date: Sat Jul 23 19:39:50.682 2005 -->
<pre>store 4293 has been terminated</pre>
<!-- Author: garfinkel -->



<!-- Date: Sat Jul 23 19:45:48.375 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12628&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12628' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12629&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12629' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12630&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12630' width='300'></a></center>
<!-- Author: Svenja -->
<!-- Comment: Sat Jul 23 19;46;31 comment by...Svenja -->
hourly plots 18:30-19:35 (5 minutes more to show end of store)

<!-- Date: Sat Jul 23 19:47:57.988 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201628">201628</A>
 Activated at 2005.07.23 19:47:24
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 19:47:58.139 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201628">201628</A>
ACTIVATE: RunSet: COSMICS_SILICON_TAPE  Trigger: COSMICS [15,619,538] with Silicon
<!-- Author: Greg x2080 -->



<!-- Date: Sat Jul 23 19:52:29.294 2005 -->
<pre>Scaler Mon tells me that the dynamic prescale function is already in use and owned by partition 16. I don't know why it does that.</pre>
<!-- Author: markus -->
<!-- Comment: Sat Jul 23 20;12;00 comment by...markus -->
<pre>I should add that this occured before I started the EVB tests and is therefore unrelated, but the last time I saw this error message the shift crew had trouble getting a physics run started later that night. I believe the solution was to restart scalerMon.</pre>
<!-- Comment: Sat Jul 23 20;55;08 comment by...markus -->
<pre>stopped scalerMon, killed the scalerMon process running on b0dap47 and started scalerMon again.</pre>
<!-- Date: Sat Jul 23 20:07:27.403 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625">201625</A>
fwd: <tt>TPOT1M, TPOT2M, TPOT3M are wrong on ACNET page (-931.91, -959.78, -840.03), but I supect it is due to the fact that I am doing the checklist after the run has been closed beacuse of no beam. <br>
It happened to me to see them wrong after the run end, even if during the run they were OK.</tt>
<!-- Author: Consumer Operator(Paola) -->


<!-- Date: Sat Jul 23 20:07:28.833 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625">201625</A>
monitors: <tt>Same trigger xsections, as in previuos run are out of allowed band.</tt>
<!-- Author: Consumer Operator(Paola) -->



<!-- Date: Sat Jul 23 20:08:34.662 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12631&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12631' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-12632&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-12632' width='300'></a></center>
<!-- Author: Paola -->
<!-- Comment: Sat Jul 23 20;10;21 comment by...Paola -->
L2_EM70_v4 trigger: the xsection is at the upper side of the band.


<!-- Date: Sat Jul 23 20:13:14.487 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201628">201628</A>
 Terminated at 2005.07.23 20:12:09
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 20:13:15.856 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201628">201628</A>
TERMINATE: abort it for fun ...
<!-- Author: Greg x2080 -->


<!-- Date: Sat Jul 23 20:14:08.514 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201629">201629</A>
 Activated at 2005.07.23 20:13:47
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 20:15:17.714 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201629">201629</A>
ACTIVATE: RunSet: COSMICS_SILICON_TAPE  Trigger: COSMICS [15,619,538] another short test
<!-- Author: Grumpy himself -->


<!-- Date: Sat Jul 23 20:18:51.006 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201629">201629</A>
 Terminated at 2005.07.23 20:16:30
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 20:20:16.161 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201630">201630</A>
 Activated at 2005.07.23 20:19:33
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 20:20:17.729 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201630">201630</A>
ACTIVATE: RunSet: COSMICS_SILICON_TAPE  Trigger: COSMICS [15,619,538] here we go again

<!-- Author: Grumpy himself -->


<!-- Date: Sat Jul 23 20:20:58.255 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201630">201630</A>
 Terminated at 2005.07.23 20:20:23
<!-- Author: RunControl -->


<!-- Date: Sat Jul 23 20:20:59.977 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201630">201630</A>
TERMINATE: ende 
<!-- Author: Grumpy himself -->



<!-- Date: Sat Jul 23 20:31:38.240 2005 -->
<pre>Very good EVB/Si test - Marcel is happy! 
We used a different Smartsocket suffix "eventBuilder2b" which sends halt/end transition message the SVX after the EVB has acknowledged the transition. This prevents both systems to access the VME bus at the same time. We took a short cosmic run and did ~20 HRRs - no problems found. 
Returned to old configuration.</pre>
<!-- Author: markus -->
<!-- Comment: Sat Jul 23 21;05;11 comment by...markus -->
<pre>The next scheduled EVB test is an end of store study. No need to call me if the store gets lost or if we get more quiet time.</pre>
<!-- Date: Sat Jul 23 20:39:11.638 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201631">201631</A>
TERMINATE: end mmmh 
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 20:43:48.080 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201632">201632</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [16,623,539] 
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 20:44:55.628 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201632">201632</A>
TERMINATE: test successful
<!-- Author: Greg (x2080) -->



<!-- Date: Sat Jul 23 20:55:47.329 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201633">201633</A>
ACTIVATE: RunSet: CALIB_RP_SMQIE Begin Roman Pots Calibration 
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 20:55:48.522 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201633">201633</A>
TERMINATE: end roman pots calibration
<!-- Author: Greg (x2080) -->



<!-- Date: Sat Jul 23 21:06:34.405 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201634">201634</A>
ACTIVATE: RunSet: ACE_TOF_QIE_NEW Calibration for TOF
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:06:35.849 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201634">201634</A>
TERMINATE: end TOF calibration
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:10:09.919 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201635">201635</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [16,623,539] Test Run
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:13:45.578 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201635">201635</A>
TERMINATE: End L2 torture run, it seems that everything still works after the experts did their tests.  
<!-- Author: Greg (x2080) -->



<!-- Date: Sat Jul 23 21:13:54.438 2005 -->
Nils has completed his XFT test and does not need to be called if the beam is lost.
<!-- Author: garfinkel -->

<!-- Date: Sat Jul 23 21:20:12.718 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201636">201636</A>
ACTIVATE: RunSet: SMQIECALIB SMX calibration
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:20:13.929 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201636">201636</A>
TERMINATE: end calibration
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:25:12.032 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201637">201637</A>
ACTIVATE: RunSet: COTCALIB_ACE Start COT calibration
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:25:13.671 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201637">201637</A>
TERMINATE: End COT Calibrations

<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:33:05.183 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201638">201638</A>
ACTIVATE: RunSet: CALIB_MUON_ALL1 Muon Calibrations
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:33:06.280 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201638">201638</A>
TERMINATE: End Muon Calibrations
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:35:10.382 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201639">201639</A>
ACTIVATE: RunSet: TOF_TAC_CALIB begin calibrations
<!-- Author: Greg (x2080) -->



<!-- Date: Sat Jul 23 21:35:20.699 2005 -->
<pre>HEP shot setup for store 4295 is in progress</pre>
<!-- Author: garfinkel -->

<!-- Date: Sat Jul 23 21:45:44.805 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201639">201639</A>
TERMINATE: end TOF TAC calibration
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 21:50:40.783 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201640">201640</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [16,623,539] 
<!-- Author: Greg (x2080) -->



<!-- Date: Sat Jul 23 22:02:18.340 2005 -->
We got a gas alarm because the IMU Supply flow rate is at the lower limit (7.0 scfh). The alarm was silenced by cryo.
<!-- Author: Svenja :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201640' target=_top>201640</a>) -->
<!-- Comment: Sat Jul 23 22;07;47 comment by...Svenja -->
the rate is back to normal (8.2 scfh)


<!-- Date: Sat Jul 23 22:17:22.378 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-12633&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-12633  width='300' align='top'></a></td><td>  
</td></tr></table>
<!-- Author: Paola -->
<!-- Comment: Sat Jul 23 22;25;57 comment by...Paola -->
After the SMX calibration, CES strips/wires bad channels plots
shows a group of flagged aligned channels. They are not 
a block of 32 channels, however I put the plot to let you know.
<!-- Date: Sat Jul 23 22:21:33.206 2005 -->
Done Timeout b0cot06, Auto HRR successful
<!-- Author: Greg :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201640' target=_top>201640</a>) -->



<!-- Date: Sat Jul 23 22:27:11.035 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-12634&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-12634  width='300' align='top'></a></td><td>  
</td></tr></table>
<!-- Author: Paola -->
<!-- Comment: Sat Jul 23 22;29;40 comment by...Paola -->
After TOF TAC calibration a white square appears in each left plot.

<!-- Date: Sat Jul 23 22:32:54.877 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201640">201640</A>
TERMINATE: end L2 torture run => ProcMon Warning about the Look Area
<!-- Author: Greg (x2080) -->



<!-- Date: Sat Jul 23 22:44:32.613 2005 -->
ISL trip in ISL B0 W3. I tried to reset the trip, but that wasn't successful. The PS GUI said that the handshake was not successful and that the wedge could not be turned on. Then it showed that this wedge tripped again because of ISL B0 W3 AVDD@ overcurrent. Silicon is paged.
<!-- Author: Svenja -->
<!-- Comment: Sat Jul 23 23;35;17 comment by...Marcel -->
Ladder f032 was not recoverable. I took it out of the Hardware Database. For details see the Silicon <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=258&frame=2&anchor=23:32:43'>
here </a>
<!-- Date: Sat Jul 23 22:44:58.999 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-12637&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-12638  width='300' align='top'></a></td><td>Look Area warning flag turned red on the ProcMon Display.  This is the 24 hour history.  There are 37 files there, and the ProcMon warning reports there should be no more than 30.  I asked our Scico to page CSL expert again.</td></tr></table>
<!-- Author: Greg -->
<!-- Comment: Sat Jul 23 23;23;18 comment by...Greg -->
No response from CSL Pager, Scico paged a second time.  There are now 38 files waiting
<!-- Date: Sat Jul 23 22:52:23.580 2005 -->
<pre>RTSERVER Error on PROCMON display.  I ran a check on RTserver and found 244 processes running.  10 minutes later, a check revealed that 243 processes were running, so I decided to kill it, following the "what to do" instructions.   
It restarted itself and after a few minutes, a check revealed only 214 processes running and the error cleared.</pre>
<!-- Author: Greg -->


<!-- Date: Sat Jul 23 22:59:33.435 2005 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2005.07.23</td><td>22:56:07</td><td>W Inner BLM</td><td>0.35</td><td>RADS</td></tr> 
<tr><td>2005.07.23</td><td>22:56:07</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2005.07.23</td><td>22:56:07</td><td>E Inner BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2005.07.23</td><td>22:56:07</td><td>E Outer BLM</td><td>0.04</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong>
<!-- Author: Svenja -->


<!-- Date: Sat Jul 23 23:01:32.262 2005 -->
<pre>final protons are being injected for store 4295</pre>
<!-- Author: garfinkel -->

<!-- Date: Sat Jul 23 23:13:25.953 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201641">201641</A>
ACTIVATE: RunSet: L2TORTURE  Trigger: L2_TORTURE [16,623,539] 
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 23:13:26.756 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201641">201641</A>
TERMINATE: end for shot setup
<!-- Author: Greg (x2080) -->


<!-- Date: Sat Jul 23 23:17:26.747 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201642">201642</A>
ACTIVATE: RunSet: AAA_SHOTSETUP  Trigger: PHYSICS_3_06 [1,683,585] 
<!-- Author: Greg (x2080) -->





<!-- Date: Sat Jul 23 23:33:38.491 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-12639&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-12640  width='300' align='top'></a></td><td>Here is another plot, only for the past 7 days.  Clearly there is something broken with the process that removes files from the look area.  Our Scico and Ops Manager are having a hard time contacting the experts</td></tr></table>
<!-- Author: Greg -->



<!-- Date: Sat Jul 23 23:42:04.849 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-12643&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-12644  width='300' align='top'></a></td><td>Bizzare.  The past 24 hour plot for the Look area dramatically changed in the past several minutes.  I don't know how this is possible.  We still have not heard from the CSL experts.  They have been paged 3 times and called at home.   
The ProcMon display still shows 38 files in the look area - but this plot certainly does not.  Very strange.</td></tr></table>
<!-- Author: Greg -->

<!-- Date: Sat Jul 23 23:54:00.688 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201642">201642</A>
TERMINATE: End to take data!
<!-- Author: Collin (x2080) -->


<!-- Date: Sat Jul 23 23:55:20.075 2005 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201607>
201607</A> 
x31387</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_06 [1,683,585]</TD>                                       
<TD ALIGN=CENTER>19:20:11</TD>                                          
<TD ALIGN=CENTER>17:28:31</TD>                                          
<TD ALIGN=CENTER>21:10:51</TD>                                          
<TD ALIGN=RIGHT>  1,553,924,272</TD>                                    
<TD ALIGN=RIGHT>    18,847,683</TD>                                     
<TD ALIGN=RIGHT>   5,213,533</TD>                                       
<TD ALIGN=RIGHT> 2919.615</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201624>
201624</A> 
x31398</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_06 [1,683,585]</TD>                                       
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>17:30:51</TD>                                          
<TD ALIGN=CENTER>::</TD>                                                
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>0</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=201625>
201625</A> 
x31399</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_06 [1,683,585]</TD>                                       
<TD ALIGN=CENTER>17:34:34</TD>                                          
<TD ALIGN=CENTER>19:20:34</TD>                                          
<TD ALIGN=CENTER>01:45:32</TD>                                          
<TD ALIGN=RIGHT>     79,590,524</TD>                                    
<TD ALIGN=RIGHT>       942,582</TD>                                     
<TD ALIGN=RIGHT>     289,034</TD>                                       
<TD ALIGN=RIGHT>  115.192</TD>                                          
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>23:55:00</TD>                                                       
<TD ALIGN=CENTER>22:56:24</TD>                                          
<TD ALIGN=RIGHT>   1,633,514,796</TD>                                   
<TD ALIGN=RIGHT>     19,790,265</TD>                                    
<TD ALIGN=RIGHT>     5,502,567</TD>                                     
<TD ALIGN=RIGHT> 3034.808</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Sat Jul 23 23:58:30.285 2005 -->
<font color=red>Shift Summary: </font><pre>scraping for store 4295</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   201607-201625
Delivered Luminosity   226.9  
Acquired Luminosity    212.2  
Efficiency             93.5
   
</td></tr></table></pre>
<!-- Author: garfinkel -->


<!-- Date: Sun Jul 24 00:25:44.983 2005 -->
<table border=0><td><tr><td bgcolor=#e0e030><h1>New Record Efficiency Store: 94.6% </h1> <ul> <b>  
<li>Store 4293: 7/22 19:17 - 7/23 19:20 
<li>Delivered Luminosity 3.2064 pb<sup>-1</sup> <li>Acquired Luminosity 3.0346 pb<sup>-1</sup> <li> Efficiency 94.6% <li> Note: >1 pb <sup>-1</sup> Acquired Luminosity required </b> </ul> <i><b>Excellent job evening, owl, day, and then evening (again) shift crews, and Congrat!!</b></i></td></tr></td></table> <font color=red><cite>(entry outside this shift's time range )</cite></font> <font color=red><cite>(entry outside this shift's time range )</cite></font>
<!-- Author: Sungwon -->