<!-- Title: Owl shift. Wed Mar 16, 2005 -->
<!-- SciCo: Giovanni Pauletta -->
<!-- DAQAce: Pedro Fernandez -->
<!-- MonAce: Xavier Portell -->
<!-- CO: Simone Donati -->
<!-- OpManager: Sungwon Lee -->
<!-- Notes: -->
Store 4044 colliding 
Run195309 running 
 
 
Date	Time	Quantity 
2005.03.16	00:02:25	C:B0ILUM	49.81	E30 
2005.03.16	00:02:25	C:B0LLUM	47.11	E30 
2005.03.16	00:02:25	C:B0PLOS	565.82	Hz 
2005.03.16	00:02:25	C:B0ALOS	196.30	Hz 
2005.03.16	00:03:54	TEVpr	8045.85	E09 
2005.03.16	00:03:54	TEVpb	1072.63	E09  


<!-- Date: Wed Mar 16 00:10:56.902 2005 -->
<pre>L2TO: HRR fixed it.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<pre>00:24 again</pre>
<pre>00:46 again</pre>
<pre>02:02 again</pre>
<pre>02:16 again.</pre>
<pre>02:25 again</pre>
<pre>02:32 again</pre>
<pre>02:49 again</pre>
<!-- Date: Wed Mar 16 00:15:29.331 2005 -->
<pre>ProcMon error. Had to restart TomCat according to instruction on web page.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 00:20:39.197 2005 -->
NB!  CORRECTION TOSTART SHIFT NOTES: 
Run 195343 is running (Not 195309)
<!-- Author: G.P. :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->



<!-- Date: Wed Mar 16 00:28:25.710 2005 -->
<pre>Done TO for b0cot11. HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 00:42:54.476 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4295&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4295' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4296&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4296' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4297&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4297' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 00;53;41 comment by...Xavier -->
<pre>Hourly plots. Note that variable LDM1GV, which thanks to an e-mail that Sebastian send to the ACE's, I could recall that it is the Tevatron Longitudinal Damper Bias, has made an step. I don't know if it is important or not. I would guess that not but there are no instructions anywhere. I will keep an eye on it anyway.

Note: it is "funny" that we are monitoring variables that we don't understand and that we don't know even which levels are the appropriate for them (except the most crucial ones). It is even funnier that we have a long list of variables here in a paper that are not the ones that we monitor and also that this particular variable is not in any document I could have found... Neither in the "big collection"! Did I miss to look somewhere?</pre>
<!-- Comment: Wed Mar 16 07;06;07 comment by...ronmoore -->
The longitudinal damper gain is being reduced gradually throughout a store to minimize (hopefully) the possibility of the damper driving beam out of the buckets and into the abort gap.  (Note T:LDM1GV = -5 means full ON, 0 = OFF.)  This damper really is needed early in a store with shorter bunch lengths (to prevent longitudinal instabilities), but it's not really necessary as the bunches become longer.
<!-- Comment: Wed Mar 16 08;51;43 comment by...Marcel -->
There is also Ricks detailed ACNET variable Page
<a href='http://www-cdfonline.fnal.gov/ops/acnet/ACNET_beamquality.html'> here </a>. I also explained this in detail in the last ACE training. And the slides of the training are not hidden, they are available on Webtalks. Third, there is always the possibility to ask people (e.g. SPL's)  either in person or by sending an email. 
<!-- Date: Wed Mar 16 00:49:43.814 2005 -->
<pre>Trigger inhibit set from SVX01 Trip.  
Run halted. 
SVX alarm.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 00;51;21 comment by...Pedro -->
<pre>Trigger inhibit:
SVX00 Trip   [000]
SVX01 Trip   [001]
SVX HV:iFix [128]
</pre>



<!-- Comment: Wed Mar 16 01;40;27 comment by...Pedro -->
<pre>Related:

 (MLE) b0dap73.fnal.gov:Thread-14255:Mar 16, 2005 1:07:34 AM->SVXMon no heart beat since 2005-03-16 00:46:08.0
 (MLE) b0dap73.fnal.gov:Thread-14255:Mar 16, 2005 1:14:35 AM->SVXMon not connected since 2005-03-16 01:13:43.0

SVXMon restarted by CO.</pre>
<!-- Date: Wed Mar 16 01:00:20.045 2005 -->
<pre>Silicon tripped. SVX VBIAS B2W2L4, DVDD B2W2L4 and AVDD B2W2L4. 
I followed the instructions to recover them but when I did, all bunch of different cells became red!!! We are paging the Si...</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 01:13:42.044 2005 -->
<table><tr><td><a href=/cgi/elog/elog.pl?nb=2005&action=view&page=-4298&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/elog.pl?nb=2005&action=view&page=-4298  width='300' align='top'></a></td><td>Silicon Trip and posterior recovery failure.</td></tr></table>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 08;54;51 comment by...Marcel -->
A careful look would have told you, that all the reds are in one bulkhead. Giving this information to the Pager Carrier may hava already made a difference.
<!-- Date: Wed Mar 16 01:23:51.548 2005 -->
<pre>SVX expert arrived. 
Everything seems to be fine now. 
SVX recovered, Run recovered, continuing data taking.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->



<!-- Date: Wed Mar 16 01:42:11.345 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4299&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4299' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4300&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4300' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4301&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4301' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 01;43;00 comment by...Xavier -->
Hourly plots. All look smooth.


<!-- Date: Wed Mar 16 01:44:24.033 2005 -->
<b>Silicon Trip</b><br> 
What actually happened was this:<br> 
e224 tripped off as it is wont to do of late.<br> 
The monitoring ace followed (kind of) the instructions on the transparency taped to the desk: 
<ul> 
<li>He used the radio buttons to switch all of SVX bulkhead 2 OFF 
<li>He used the same one to switch it all back ON 
</ul> 
The transparency indicates that the entire power supply be turned off and back on (which is bad), and what he did is to turn the entire bulkhead off and back on (which is worse).  It is admittedly not clear from the transparency where the arrows are pointing, but it is bad thing to ever be turning any silicon completely off (as opposed to standby) without some very good reason (one ladder tripping is not such a reason, you should have an expert watching over your shoulder before doing such an operation).  <p> 
So a large chunk of ladders tripped off on DVDD overcurrent on the "OFF" command.  They were recovered by the subsequent "ON" command.<p> 
<b> The easiest proper way to recover a silicon trip iff the silicon is ON is to go to the main IFIX page and click the up/on button for the silicon. If the silicon trips from standby click the diamond-shaped standby button.  This will always do what you want.  <p> 
If you really are sure what you are doing (if you don't, do option 1 and/or page us), you can select the wedge in question in the power supply gui (using the detector cartoon at the bottom), and then use the buttons in the <u>upper left hand corner</u> to bring the selected ladder from off, to standby, to on.</b>
<!-- Author: corrinne :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 02;20;03 comment by...Xavier -->
My humble suggestion is that when doing training shifts, it could be a good idea to make a "hands-on" session where we can see the display and the different normal procedures that we can do if problems appear and warnings about what we don't need to do. I do think the important ideas stick better talking in front of the real program. We are asked to use common sense but even common sense need to be trained to understand the environment. 
<!-- Comment: Wed Mar 16 06;56;59 comment by...Silicon Emeritus #3 -->
Historical note:<P>
The very good reason for power cycling the entire wedge/power supply was that the most common trip, on DVDD, often required it,
so to keep the Ace instructions simple we made that the default (and only) action.<P>
The trickiness with the radio buttons is that the GUI sometimes
get confused about which button was clicked.  That is why there
is a confirmation dialog <i>which must be checked</i> or you turn off (or worse, ON) things you didn't mean to.<P>
I didn't know the new policy was to use the IFIX buttons, which at one point was terribly slow.  If so, I certainly agree that the instructions should be updated. I also agree that a 5 minute hands on session is a very good idea.  Maybe someone can fix the PS GUI feature too, although several former SPLs have begged for this to no avail. 


<!-- Comment: Wed Mar 16 07;33;15 comment by...corrinne -->
Thanks, "silicon emeritus", for the explanation about cycling the entire wedge!  The problem happens when things start tripping off, and because of the ambiguity in the posted instructions about which buttons to use, along with the usual PSGUI issues.  That is why I've seen other experts recommend using the IFIX buttons, and why I chose to recommend it myself.  No, I would not say it the official policy, and yes, it is slow, but if the ace is at all confused it is really the safest option, I think.
<!-- Comment: Wed Mar 16 08;39;38 comment by...Cowardly Ex-Ace -->
<pre>Many moons ago this same scenario happened to this anonymous ace.  I tried to follow the cartoon diagram of how to recover a tripped silicon ladder, and ended up turning off and on an entire side of the silicon due to confusion over where the arrow on the cartoon was pointing.   So, hands-on training for these situations would most certainly be beneficial to the silicon and to the shift crew :-)</pre>
<!-- Comment: Wed Mar 16 09;06;26 comment by...by a current SPL  -->
Well, there was a kind of hands on session in the last ACE training, showing them the GUI and how to operate it.  But experience teaches us:
<ul
<li> The amount of information you can cram into ACe's heads is limited in two hours. So that would require more training sessions, fine for us, if that is fine for you</li>
<li>The silicon is too complex to cover each and every failure mode in a procedure, at some point use your common sense and if you are not sure, page Silicon <u>before</u> you do anything</li>
</ul>

The powering by wedge has good reasons, as already mentioned above and as already been pointed, reading the confirmation messages is a good idea. 
<!-- Date: Wed Mar 16 02:06:16.597 2005 -->
<pre>Another trip of the same cells. The expert was still here so she supervised the operation. All fine. :)</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 02:09:22.432 2005 -->
<pre>Trigger inhibit set from SVX01 Trip.   
Run halted. 
SVX alarm. 
 
SVX recovered. 
Run recovered. 
Continuing data taking.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 02;37;35 comment by...Pedro -->
<pre>02:32 again</pre>
<!-- Date: Wed Mar 16 02:36:56.546 2005 -->
SVX trip again for the same cells as before. Now I recovered it using the HV ON button. It was successful. Maybe it take a little bit longer.
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->




<!-- Date: Wed Mar 16 02:39:52.367 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4302&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4302' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4303&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4303' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4304&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4304' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 02;40;18 comment by...Xavier -->
Hourly plots. All smooth.



<!-- Date: Wed Mar 16 02:59:56.364 2005 -->
<pre>L2 Decision TO. 
Trigger Inhibit Set from TOF HV:IFix [132]. 
auto-HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 03:05:57.379 2005 -->
<pre>Consumer error:  
CER_SVXMON_HALT_RECOVER_RUN_ERROR 
Stuck Cellid S/B2/W7/L2/C0-4. 
 
AUTO HRR fixed it. 
</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 03:11:02.588 2005 -->
<pre>L2 Decision TO. HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 03;27;32 comment by...Pedro -->
<pre>03:22 again.</pre>


<!-- Date: Wed Mar 16 03:30:10.664 2005 -->
<pre>Got heartbeat alarms from ICI, CLC, BSC, CPR, RPS and MNP. We waited and they cleared (ICI is not clear yet but it seems no stuck when looking its outputs). 
 
I think there should be a way to know which is the frecuency that every system is taking to read again. One minute, two minutes... It is just worthy to know how long we should wait before doing something else.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 03;41;02 comment by...Xavier -->
Same HeartBeat alarm after a while all is recovered.
<!-- Date: Wed Mar 16 03:37:59.048 2005 -->
<pre>Done TO for b0cot11: AutoHRRed 
 
Details: 
(MLE) b0cot11:Messenger:Mar 16, 2005 3:33:41 AM->TDC Invalid SRAM Header evt 4196040 slot 15: exp/fnd mod 195/60 l2b 0/0 1E000025 
(MLE) b0cot11:Messenger:Mar 16, 2005 3:33:41 AM->TDC Header Mismatch evt 4196040 slot 15: static/fifo 0x6181260C/0x1E000025 [08C,090] 
(MLE) b0cot11:Messenger:Mar 16, 2005 3:33:41 AM->TDC evt 4196040 slot 15 static/fifo: ModID 195/60 L2B 0/0 Hits 294/0 BC 2/37  
MLE) b0cot11:Messenger:Mar 16, 2005 3:33:41 AM->Event 4196040: Bunchcounters in slot 4 (BC=12) and slot 15 (BC=37) disagree (L2B=0) 
(MLE) b0cot11:Messenger:Mar 16, 2005 3:33:41 AM->Runtime Error 34, Event 4196040: Bunch counter mismatch, mismatch count = 1 
(MLE) b0l3pcom2.fnal.gov:main:Mar 16, 2005 3:33:41 AM->Error on L3 node b0l3238 (partition 0) Wed Mar 16 03:33:40 2005 l3_node  in refoInt_reformatProc (l3_refevt.c:610) 
     ( L3_REFORMAT_ERROR: see Error Handler _reformatter.log file for full message) 
CT: 2005.03.16 03:33:45 
33'45" 1 crate/s: b0cot11(59),  Done TO. [RXPT] 
(MLE) b0dap73.fnal.gov:Thread-14255:Mar 16, 2005 3:33:45 AM->Done Timeout: COT_11 
(MLE) b0dap73.fnal.gov:Thread-14255:Mar 16, 2005 3:33:45 AM->Requested Halt-Recover-Run issued [errmon] 
</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->



<!-- Date: Wed Mar 16 03:44:11.969 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4305&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4305' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4306&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4306' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4307&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4307' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 03;45;16 comment by...Xavier -->
Hourly plots. Small peak at LOSTPB. Possibly related with when D0 put the roman pots in?


<!-- Date: Wed Mar 16 04:07:23.376 2005 -->
<pre>L2 Decision TO. AutoHRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 04;10;51 comment by...Pedro -->
<pre>04:09 again</pre>


<!-- Date: Wed Mar 16 04:30:01.530 2005 -->
<pre>-Event Builder error (04:16:34) 
SCPU_BAD_CHANNEL_COUNTS Error 
Hardware EVB has detected a problem with data quality in SCPU b0eb23. 
 
-AutoHRR failed.  
Consecutive errors:  
L3TO, Busy TO for b0svx06, L2 Trigger problems 
 
-Halt/Recover/Run successful, continuing data taking (04:18:40). 
</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 04:31:45.265 2005 -->
<pre>-Event Builder error: 
SCPU_BAD_CHANNEL_COUNTS Error 
Hardware EVB has detected a problem with data quality in SCPU b0eb17. 
 
-Auto-HRRed. 
</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 04:41:46.808 2005 -->
<pre>-Done TO for b0cot10.  
-HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 04:45:43.948 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4308&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4308' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4309&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4309' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4310&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4310' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 04;46;42 comment by...Xavier -->
Hourly plots. All fine.


<!-- Date: Wed Mar 16 04:55:57.455 2005 -->
<pre>L2TO; HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 05:19:04.816 2005 -->
<pre>04:58:55  Icicle heart beat missing  reported by RC
05:12:55  again, plus purple field in Global Alarms
Restarted Icicle 05:13.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 05;40;25 comment by...Pedro -->
<pre>05:26:56 Heart beat missing reported by RC but ICI field on Global Alarms is still green. Also ProcMon complains about missing ICI heart beat. Expert of previous shift says that it may longer then 10 min until ProcMon receives reasonable ICI response. So we decided to wait a little before doing further actions. icicle.exe is still producing output on the terminal.
</pre>
<!-- Comment: Wed Mar 16 05;49;50 comment by...Pedro -->
<pre>ICI heart beat alarm from iFix. (Global Alarms has purple ICI field). Might be related to the same incident reported by RC previously. icicle.exe is still running. So let's wait a little.</pre>
<!-- Comment: Wed Mar 16 07;37;06 comment by...Pedro -->
<pre>Since 05:25:56 no ICI heart beat alarm anymore.</pre>
<!-- Date: Wed Mar 16 05:22:56.954 2005 -->
<pre>-Trigger Inhibit from TOF HV:IFix [132] (05:09:59).  
-Auto-HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->



<!-- Date: Wed Mar 16 05:49:12.403 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4311&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4311' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4312&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4312' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4313&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4313' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 05;49;42 comment by...Xavier -->
Hourly plots. All fine.



<!-- Date: Wed Mar 16 06:05:06.574 2005 -->
<pre>Event Builder Error (05:55:12) 
 
 SCPU_CANT_RESET_VRB Error  
 THE BAD VRB CRATE is: b0eb15 
 THE BAD VRB MODULE is in SLOT: 12 
 
Halted run, pressed reset button on eb15 VRB slot 12 for 10 secs. Recovered the run. Continuing data taking (05:59:57).</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 06:07:43.558 2005 -->
<pre>Done TO for b0cot12 (06:01:58). 
Auto-HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 06:08:40.006 2005 -->
<pre>L2TO (06:06:13). 
Auto-HRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 06:26:41.529 2005 -->
<pre>Event Builder error (06:21:58); 
 SCPU_TRACER_EVENT_ID Error !!! 
 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb16 (forwarded by FER crate PCAL_09). 
 
Done TO b0pcal09; 
L2TO; 
 
Auto-HRRed; 
 
</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 06:30:03.396 2005 -->
SVX trip. Same "classical" cells. Restored successfully.
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 06;44;04 comment by...Xavier -->
It happened again... Very close this time!
<!-- Comment: Wed Mar 16 07;22;25 comment by...Xavier -->
Another one. Seems that now the Silicon is not happy...
<!-- Date: Wed Mar 16 06:30:12.175 2005 -->
<pre>L2TO (06:26:59); 
Auto-HRRed;</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->
<!-- Comment: Wed Mar 16 06;37;52 comment by...Pedro -->
<br> 06:36:25 again 
<br> 06:36:58 again
<!-- Comment: Wed Mar 16 06;43;19 comment by...Pedro -->
<br> 06:41:21 again
<br> 06:41:41 again
<!-- Comment: Wed Mar 16 06;49;03 comment by...Pedro -->
<br> 06:47:09 again

<!-- Date: Wed Mar 16 06:32:15.028 2005 -->
<pre>Trigger Inhibit set from SVX01 Trip [001]; 
After recovering of SVX, continuing data taking (06:29:04);</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->

<!-- Date: Wed Mar 16 06:33:12.525 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194800">194800</A>
COMMENT: Comment from bzora1; please ignore.
<!-- Author: badgett -->






<!-- Date: Wed Mar 16 06:44:59.344 2005 -->
<pre>Trigger Inhibit set again from SVX01 Trip [001];  
After recovering of SVX, continuing data taking (06:43:47);</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 06:46:22.353 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4314&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4314' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4315&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4315' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4316&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4316' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 06;46;59 comment by...Xavier -->
Hourly plots. All seems fine.





<!-- Date: Wed Mar 16 07:12:31.535 2005 -->
<pre>Event Builder error (07:10:35) 
 SCPU_TRACER_EVENT_ID Error 
 Hardware EVB has detected a problem with data quality in 
 SCPU b0eb16 (forwarded by FER crate PCAL_09). 
 
AutoHRRed.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->


<!-- Date: Wed Mar 16 07:22:03.804 2005 -->
<pre>Trigger Inhibit set again from SVX01 Trip [001] (07:19:30);   
After recovering of SVX, continuing data taking (07:21:30);</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->



<!-- Date: Wed Mar 16 07:28:44.481 2005 -->
<pre>From L3 display: Total Reformatter reject rate 0.02%.</pre>
<!-- Author: Pedro :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343' target=_top>195343</a>) -->




<!-- Date: Wed Mar 16 07:45:01.986 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4317&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4317' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4318&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4318' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-4319&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-4319' width='300'></a></center>
<!-- Author: Xavier -->
<!-- Comment: Wed Mar 16 07;45;43 comment by...Xavier -->
Hourly plots. All seems fine.

<!-- Date: Wed Mar 16 07:55:53.810 2005 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195343>
195343</A> 
x2FB0F</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [3,612,555]</TD>                                       
<TD ALIGN=CENTER>22:08:44</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>08:12:30</TD>                                          
<TD ALIGN=RIGHT>    688,752,991</TD>                                    
<TD ALIGN=RIGHT>     8,160,673</TD>                                     
<TD ALIGN=RIGHT>   2,141,331</TD>                                       
<TD ALIGN=RIGHT> 1263.873</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>07:55:03</TD>                                                       
<TD ALIGN=CENTER>08:12:30</TD>                                          
<TD ALIGN=RIGHT>     688,752,991</TD>                                   
<TD ALIGN=RIGHT>      8,160,673</TD>                                    
<TD ALIGN=RIGHT>     2,141,331</TD>                                     
<TD ALIGN=RIGHT> 1263.873</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Wed Mar 16 07:55:55.869 2005 -->
<font color=red>Shift Summary: </font>Begin shift or beam at 00:01:44 
End shift or beam at 07:53:51 
Delivered luminosity: 1158.21 
Acquired luminosity: 956.04 
Totals 
Date:	2005.03.16 
Shift:	owl 
Delivered luminosity:	1158.2 nb-1 
Acquired luminosity:	956.0 nb-1 
Efficiency:	82.5 
   
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   195343
Delivered Luminosity   1158  
Acquired Luminosity    956  
Efficiency             82.5
   
</td></tr></table></pre>
<!-- Author: Giovanni Pauletta -->