<!-- Title: Day shift. Wed Oct 22, 2003 -->
<!-- SciCo: Weiming Yao  -->
<!-- DAQAce: Helen Hayward -->
<!-- MonAce: &nbsp; -->
<!-- CO: &nbsp; -->
<!-- OpManager: Steve Hahn -->
<!-- Notes: -->
<pre>To fix L2, run L2_torture test, and maybe some cosmic.</pre>


<!-- Date: Wed Oct 22 08:08:25.797 2003 -->
<pre>Probably a pointless run (but since L2 is still broken and no one else is here) 
 
L2 auto accept on, COT script used, etdc00, bl2de00, b0svt03 removed. 
 
I just want to see for myself that something will work today!</pre>
<!-- Author: Helen -->

<!-- Date: Wed Oct 22 08:12:58.232 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171684">171684</A>
ACTIVATE: L2_Torture
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 08:13:00.139 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171684">171684</A>
TERMINATE: Oops, forgot to remove b0svt06
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 08:18:32.770 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171685">171685</A>
ACTIVATE: L2_Torture[13,282,370]
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 08:18:33.033 2003 -->
Followed previous ace instructions on which crates to remove, but as i try to activate the run, b0tsi00 crate shows error.  Probably due to trying to L2_Torture with no L2!
<!-- Author: Helen :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171685' target=_top>171685</a>) -->

<!-- Date: Wed Oct 22 08:22:18.420 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171685">171685</A>
TERMINATE: aborted
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 08:24:28.790 2003 -->
The error was actually found to originate from b0xft00.  This crate has been rebooted so we can try again.
<!-- Author: Helen -->
<!-- Comment: Wed Oct 22 08;29;46 comment by...Helen -->
This didn't work
<!-- Date: Wed Oct 22 08:25:46.171 2003 -->
remove cot script to see if it is the cause of the errors....
<!-- Author: Helen -->

<!-- Date: Wed Oct 22 08:27:57.560 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171687">171687</A>
ACTIVATE: L2_Torture, without the COT script
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 08;28;36 comment by...Helen -->
Taht worked but we then got L2 trigger rate dropping to 0Hz



<!-- Date: Wed Oct 22 08:46:47.812 2003 -->
<pre>We tried using setcv script to lower the DAC threshold. 
Data volume was too large for the VRB buffers and we got  
a timout right away. 
 
 
 
</pre>
<!-- Author: Frank Chlebana -->


<!-- Date: Wed Oct 22 08:57:22.951 2003 -->
<table border=0><td><tr><td bgcolor=#e0e030><h2><b> Cooling Trip Story </b></h2></td></tr></td></table>
<!-- Author: Rainer :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER= ' target=_top> </a>) -->
<!-- Comment: Wed Oct 22 09;08;45 comment by...Rainer -->
Frequent cooling trips were caused by a flaky connection, 
diagnosis
<a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=171&frame=2&anchor=20:14:11'>
here.</a> The cable in question was strain relieved and a faulty RTD sensor fixed, so we expect this problem to be taken care. In case of further cooling trips  I repeat standard procedure here: <p>
page the silicon cooling expert at <b>218-8626 </b><BR>
<b>AND</b> <BR>
page the main silicon pager at <b>218-8227 </b> <BR>
<p>
In addtion (this is not standard procedure) try to get a hold of people working in the bore who  likely touched a cooling line such  that they at least know. The EAST bore phone 6787, WEST bore phone is 6785 or try to page them via the intercom. <p>
If in doubt, don't hesitate  to contact the main silicon pager carrier who actually sits most of the time in the silicon office  these days. cooling alarms are potentially serious conditions so they do merit our attention.



<!-- Date: Wed Oct 22 09:02:41.047 2003 -->
Tomcat appears to have been succesfully restarted... although we are still waiting for a green box from ProcMon.
<!-- Author: Helen -->



<!-- Date: Wed Oct 22 09:12:05.538 2003 -->
All CCAL, PCAL, WCAL have been released for expert to use.
<!-- Author: Helen -->


<!-- Date: Wed Oct 22 09:18:22.626 2003 -->
<pre>After a cleanup of the evb, the following crates required reseting: 
 
eb13, eb14,eb15,eb25. 
 
Also, ProcMon was restarted.  The following boxes were red:   
 
DAQ VOICE ALARMS 
SVT SPY MONITOR 
CSL_MAIN 
DOWNTIME ENTRIES  
FILESYSTEM 
 
The "check" for the above crates has been disabled, so they now show grey. </pre>
<!-- Author: Helen -->


<!-- Date: Wed Oct 22 09:24:16.550 2003 -->
<pre>I removed all COT crates excecpt for COT02 which has  
Rev F TDCs in slots 12-16. 
 
Lowered DAC thresholds and started a run. Ran for a bit and 
got TRACER FIFO half full errors. This is because we are trying to push through too much data with the lowered thresholds. 
 
Right after that we got a error from cot02 crate and we get  
stuck in the HRR (RUN) transition - TSI00 never returns active 
because the COT02 crate is pulling error. 
 
Only option is to abort the run. 
 
</pre>
<!-- Author: Frank Chlebana :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171695' target=_top>171695</a>) -->

<!-- Date: Wed Oct 22 09:29:34.679 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171696">171696</A>
ACTIVATE: QIE calorimeter calibration
<!-- Author: spezziga -->



<!-- Date: Wed Oct 22 09:29:50.262 2003 -->
<pre>Attempting to see what works with L2_Torture.   
 
First attempt, remove only booked CAL crates.</pre>
<!-- Author: Helen -->

<!-- Date: Wed Oct 22 09:39:36.998 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171697">171697</A>
ACTIVATE: L2_Torture
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 09:46:23.187 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171696">171696</A>
TERMINATE: QIE calibration downloaded in all calorimeter crates.
Note: channels connected to PHA pin diodes have wrong pedestal values.

<!-- Author: spezziga -->
<!-- Comment: Wed Oct 22 09;48;39 comment by...m mattson -->
<br>ACEs will no longer see "ADMEM not configured" for ccal crates, at the beginning of the run.  The note about PIN diodes applies for PHA channels 0 and 1, which usually swamp the "bad channel" list for the calibration.

<!-- Date: Wed Oct 22 09:50:36.063 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171698">171698</A>
ACTIVATE: L2_Torture
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 09:58:04.416 2003 -->
Twice when we have been investigating the L2 problem, we have had to reboot, reset and shepherd crate b0svt05 beacause it showed an error when trying to configure.
<!-- Author: Helen -->

<!-- Date: Wed Oct 22 09:58:55.726 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171699">171699</A>
ACTIVATE: L2_Torture, with HEVB turned off and without L3
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 10;00;19 comment by...Helen -->
This tie we managed to get a few thousand events before the first L2 decision time-out errors

<!-- Comment: Wed Oct 22 10;00;47 comment by...Helen -->
sorry typo: 'time' not 'tie'

<!-- Date: Wed Oct 22 10:04:59.754 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171699">171699</A>
TERMINATE: end 
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:10:20.734 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171700">171700</A>
ACTIVATE: L2_Torture, without HEVB or L3 again.  All Consumers are on.
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 10:10:21.375 2003 -->
I reconnected a L1 cable on pcal01, slot 15, output 5.  (Carla P mentioned she saw the occasional glitch, and thought we should check the connection.)  It looked okay beforehand, but took the action anyways.<p> 
There was a problem I saw during Monday's checkout, which has gone away on its own.  Pcal07, slot 19, ch 1 was not distributing events evenly among its capacitors, but it looks fine now.
<!-- Author: m mattson -->


<!-- Date: Wed Oct 22 10:16:18.493 2003 -->
While the ACE was restarting the Consumer Displays, we noticed that selecting "LumMon" brings up the TrigMon plots, and vice versa.  Go figure...
<!-- Author: m mattson and random people in control room -->

<!-- Date: Wed Oct 22 10:18:25.172 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171700">171700</A>
TERMINATE: end run
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:26:26.871 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171701">171701</A>
ACTIVATE: PHYSICS_1_05[8,345,405]: obviously with no beam
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:30:02.247 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171701">171701</A>
TERMINATE: abort run due to b0svt05 not activating
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:32:41.371 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171702">171702</A>
ACTIVATE: PHYSICS_1_05[8,345,402]
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 10;33;22 comment by...Helen -->
b0svt05 has been removed and rebooted.  I will add it in again for the next run.

<!-- Comment: Wed Oct 22 10;59;38 comment by...Helen -->
It was added into a L3 test run and worked fine.
<!-- Date: Wed Oct 22 10:36:18.954 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171702">171702</A>
TERMINATE: end run to remove COT crates because they are requested by an expert
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:49:21.507 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171703">171703</A>
ACTIVATE: PHYSICS_1_05
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:50:02.497 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171703">171703</A>
TERMINATE: end run to release b0etdc00 for expert
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 10;52;08 comment by...Helen -->
also released b0htdc00 crate

<!-- Date: Wed Oct 22 10:54:16.252 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171704">171704</A>
ACTIVATE: PHYSICS_1_05[8,345,402]
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:57:10.436 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171704">171704</A>
TERMINATE: end run to put in myron mode
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 10:59:17.150 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171706">171706</A>
ACTIVATE: PHYSICS_1_05[8,345,402]
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 11:23:26.475 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171706">171706</A>
TERMINATE: end run
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 11:31:26.164 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171710">171710</A>
ACTIVATE: clc timing test + dcas study
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 11:40:37.666 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171710">171710</A>
TERMINATE: stop run
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 11:40:40.409 2003 -->
ProcMon started lying to us saying, so we stopped and restarted it.
<!-- Author: Helen -->

<!-- Date: Wed Oct 22 11:47:03.872 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171712">171712</A>
ACTIVATE: PHYSICS_1_05[8,345,402]:  still myron mode
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 11:58:43.200 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171712">171712</A>
TERMINATE: end run
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 11:59:10.902 2003 -->
Paged by the control room, as ccal00 and ccal07 (8??) were booked by someone else.  According to "bookcrate", they were claimed by "Max x2120".  As I was talking to the ACE, the crates were released.
<!-- Author: m mattson -->

<!-- Date: Wed Oct 22 12:03:19.425 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171713">171713</A>
ACTIVATE: PHYSICS_1_05[8,345,402]:  cal crates back in, b0xtrp00 released for expert
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 12;06;37 comment by...Helen -->
(MLE) b0dap73.fnal.gov:Thread-21345:12:03:37 PM->Done Timeout: TRIGGER_SCALERS_00<br>
solved by auto HRR.

<!-- Comment: Wed Oct 22 12;08;11 comment by...Helen -->
(MLE) b0dap73.fnal.gov:Thread-21345:12:06:17 PM->Done Timeout: TRIGGER_SCALERS_00<br>
another auto HRR

<!-- Date: Wed Oct 22 12:16:35.029 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171713">171713</A>
TERMINATE: experts request ccal00 and ccal07
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 12:16:50.548 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171717">171717</A>
ACTIVATE: XTRP Three Track Test -- checking for false 1's in rampipe beamerrorB
<!-- Author: cmarino -->


<!-- Date: Wed Oct 22 12:21:41.233 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171718">171718</A>
ACTIVATE: PHYSICS_1_05[8,345,402]
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 12:35:16.314 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171718">171718</A>
TERMINATE: end to swap dcas card
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 12:38:35.110 2003 -->
<pre>More ProcMon error. 
 
RESMGR, CONSUMER MONITOR PROXY, DBBROKER PROXY and CONSUMER STATE MANAGER all show errors.  The first three of these run on the same computer b0dap63, and all are refusing to obey commands! </pre>
<!-- Author: Helen -->
<!-- Comment: Wed Oct 22 13;15;02 comment by...Helen -->
Jane is investigating.
<!-- Date: Wed Oct 22 12:40:44.315 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171721">171721</A>
ACTIVATE: PHYSICS_1_05[8,345,402]
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 13:03:17.372 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171722">171722</A>
ACTIVATE: getting out of jane's way
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 13:07:37.598 2003 -->
Previous run had DCAS SNO 61 (back from Michigan after repairs, ah, ah) in slot 15 of b0l2cl03 in place of SNO 77. Noticed errors in CLIST Trigmon canvas. 
Stopped run and swapped that DCAS with SNO 48 (back from MIchigan after repairs)
<!-- Author: monica -->

<!-- Date: Wed Oct 22 13:08:26.536 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171722">171722</A>
TERMINATE: ending to surrender ccal
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 13:12:46.383 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171723">171723</A>
ACTIVATE: PHYSICS_1_05[8,345,402]
<!-- Author: Helen x2080 -->




<!-- Date: Wed Oct 22 13:18:31.258 2003 -->
Errors in CLIST Trigmon Cluster canvas persist. 
Note that just before run 171721, LOCOS was swapped between 
b0l2cl02 and b0l2cl03 (to track down bad ISOLIST eta=0 bit). 
Optical fiber for LOCOS in b0l2cl03 was very hard to push  
out and in....
<!-- Author: monica :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171723' target=_top>171723</a>) -->


<!-- Date: Wed Oct 22 13:28:24.683 2003 -->
Found errors in CLIST info canvas for run 171718 which was  
taken before swapping any DCAS board but AFTER swapping LOCOS. 
I will try to reseat those boards and their cables.
<!-- Author: monica -->


<!-- Date: Wed Oct 22 13:38:08.908 2003 -->
Finally fixed RESMG PROXY, CONSUMER MONITOR PROXY and DBBROKER PROXY.  The problem was with the monitoring as opposed to the actual processes we think.  CONSUMER STATE MANAGER is still red.
<!-- Author: Helen -->

<!-- Date: Wed Oct 22 13:38:53.790 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171726">171726</A>
ACTIVATE: source run ccal00

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 13:42:34.993 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171727">171727</A>
ACTIVATE: XTRP Three Track Test -- checking for false 1's in rampipe beamerrorB 
<!-- Author: cmarino -->


<!-- Date: Wed Oct 22 13:44:58.140 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171728">171728</A>
TERMINATE: abort run due to b0l2de00
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 13:44:59.455 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171726">171726</A>
TERMINATE: source run ccal00

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 13:47:36.211 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171730">171730</A>
ACTIVATE: greg reseated the alpha and we are going again
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 13:50:29.162 2003 -->
Reseating fiber for LOCOS in crate b0l2cl03 seems to have fixed the CLIST Info errors observed in TrigMon
<!-- Author: monica :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171730' target=_top>171730</a>) -->

<!-- Date: Wed Oct 22 13:52:35.511 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171733">171733</A>
ACTIVATE: source run ccal08
<!-- Author: fotis 2080 -->



<!-- Date: Wed Oct 22 13:57:30.900 2003 -->
Put DCAS SNO 61 in VME spare cabinet 3. 
Take SNO 77 back to Michigano for testing.
<!-- Author: monica -->

<!-- Date: Wed Oct 22 13:57:56.688 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171733">171733</A>
TERMINATE: source  run ccal08

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 14:03:50.223 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171737">171737</A>
ACTIVATE: source run ccal09
<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 14:10:24.268 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171730">171730</A>
TERMINATE: end run
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 14:10:25.377 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171737">171737</A>
TERMINATE: source run ccal09
<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 14:15:35.428 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171739">171739</A>
ACTIVATE: source run ccal09

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 14:20:11.118 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171739">171739</A>
TERMINATE: source run ccal09

<!-- Author: fotis 2080 -->



<!-- Date: Wed Oct 22 14:20:12.005 2003 -->
<pre>Aim to run L2 torture with whatever crates are left over from everyone else: 
 
CCAL_09 
COT_00 to COT_19 
COT_PULSER_00 
XFT_XTRP_00 
 
I will stop and add in COT crates as soon as they are available.</pre>
<!-- Author: Helen -->
<!-- Comment: Wed Oct 22 14;28;01 comment by...Helen -->
Add all the CCAL crates to that list!
<!-- Date: Wed Oct 22 14:27:44.493 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171741">171741</A>
TERMINATE: Oops!  I booked the Calorimeter crates and people wanted them! my mistake. 
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 14:30:12.851 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171742">171742</A>
ACTIVATE: L2_Torture with left overs!
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 14:31:33.293 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171742">171742</A>
TERMINATE: abort due to not activating
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 14;33;22 comment by...Helen -->
<pre>Highlights of error log:

(RC)   14:28:41 Config -&gt; READY
(MLE) b0tsi00:Messenger:2:28:49 PM-&gt;TS L2 FSM failed to start (State 14)
CT: 2003.10.22 14:30:11
30'11" 1 crate/s: b0xft00(97),  in error.[RXPT]b0dap73.fnal.gov:Thread-21345:2:30:11 PM-&gt;Succes
sfully shepherded client XFT_FINDER_00 [b0xft00][MLE]</pre>
<!-- Date: Wed Oct 22 14:31:50.230 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171745">171745</A>
ACTIVATE: source run ccal08

<!-- Author: fotis 2080 -->



<!-- Date: Wed Oct 22 14:37:18.366 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171745">171745</A>
TERMINATE: source ccal08

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 14:39:01.611 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171747">171747</A>
ACTIVATE: XTRP Three Track Test -- checking for false 1's in rampipe beamerrorB


<!-- Author: cmarino -->


<!-- Date: Wed Oct 22 14:41:46.708 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171752">171752</A>
ACTIVATE: source run ccal08

<!-- Author: fotis 2080 -->



<!-- Date: Wed Oct 22 14:45:57.693 2003 -->
I tried rebooting b0xft00, but the same errors persist.  I am now trying L2_Torture excluding this crate.
<!-- Author: Helen -->


<!-- Date: Wed Oct 22 14:46:40.623 2003 -->
Cryo informed S1 will be down all night due to software problem. 
<!-- Author: weiming -->

<!-- Date: Wed Oct 22 14:48:23.877 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171753">171753</A>
ACTIVATE: L2_Torture [13,282,370] with left over crates and b0xft00 excluded
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 14;48;49 comment by...Helen. -->
At last this appears to be working fine.
<!-- Date: Wed Oct 22 14:48:24.568 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171752">171752</A>
TERMINATE: source run ccal08

<!-- Author: fotis 2080 -->



<!-- Date: Wed Oct 22 14:50:15.571 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171753">171753</A>
TERMINATE: end run to try including crate b0xft00 again
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 14:51:58.349 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171755">171755</A>
ACTIVATE: XTRP Three Track Test -- checking for false 1's in rampipe beamerrorC


<!-- Author: cmarino -->


<!-- Date: Wed Oct 22 14:56:44.409 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171758">171758</A>
ACTIVATE: source run ccal07
<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 14:56:45.812 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171757">171757</A>
ACTIVATE: just checking b0xft00 error is still here!
<!-- Author: Helen x2080 -->
<!-- Comment: Wed Oct 22 14;58;29 comment by...Helen -->
yup: definent problem with this one.
<!-- Date: Wed Oct 22 14:57:10.318 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171757">171757</A>
TERMINATE: aborted
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 15:01:09.207 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171759">171759</A>
ACTIVATE: L2_TORTURE[13,282,370] with leftovers and b0xft00 excluded again
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 15:01:10.690 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171758">171758</A>
TERMINATE: source run ccal07

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 15:07:28.312 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171762">171762</A>
ACTIVATE: source run b0ccal06
<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 15:10:42.767 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171764">171764</A>
ACTIVATE: source run ccal06

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 15:14:44.501 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171767">171767</A>
ACTIVATE: source run ccal 00
<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 15:20:07.679 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171767">171767</A>
TERMINATE: source run b0ccal00

<!-- Author: fotis 2080 -->


<!-- Date: Wed Oct 22 15:22:14.650 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171770">171770</A>
ACTIVATE: source run b0ccal00
<!-- Author: fotis 2080 -->



<!-- Date: Wed Oct 22 15:22:58.641 2003 -->
(MLE) b0cmx00:Messenger:3:19:19 PM->Runtime Error 5, Event 469807: No valid header for TDC in slot 13 
CT: 2003.10.22 15:19:20 
19'20" 1 crate/s: b0xft02(99),  in error.[RXPT]<br> 
(MLE) b0dap73.fnal.gov:Thread-21345:3:19:20 PM->FrontEnd Crate Error Condition from: XFT_FINDER_02<br> 
<br> 
(MLE) b0cmx00:Messenger:3:19:19 PM->Runtime Error 6, Event 469807: No valid header for TDC in slot 13<br> 
(MLE) b0dap73.fnal.gov:Thread-21345:3:19:20 PM->FrontEnd Error: XFT_FINDER_02<br> 
(MLE) b0dap73.fnal.gov:Thread-21345:3:19:20 PM->Requested Halt-Recover-Run issued [errmon]<br> 
<br> 
<br> 
Solved itself.<br>
<!-- Author: Helen :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171759' target=_top>171759</a>) -->

<!-- Date: Wed Oct 22 15:26:49.595 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171770">171770</A>
TERMINATE: source run ccal00

<!-- Author: fotis 2080 -->



<!-- Date: Wed Oct 22 15:26:49.811 2003 -->
Hmm. two more errors that required auto HRR within a few minutes of each other.<br> 
<br> 
(MLE) b0dap73.fnal.gov:Thread-21345:3:20:54 PM->Done Timeout: CLC_00 EM_TIMING_00<br> 
(MLE) b0dap73.fnal.gov:Thread-21345:3:20:54 PM->FrontEnd Crate Error Condition from: 
XFT_FINDER_02<br> 
<br> 
Then later:<br> 
<br> 
22'12" 1 crate/s: b0xft02(99),  in error.[RXPT]<br> 
(MLE) b0dap73.fnal.gov:Thread-21345:3:22:12 PM->FrontEnd Crate Error Condition from: 
XFT_FINDER_02<br> 
<br> 
 
I'll see if it sorts itself out.....<br>  
<!-- Author: Helen :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171759' target=_top>171759</a>) -->

<!-- Date: Wed Oct 22 15:33:09.146 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171759">171759</A>
TERMINATE: end run to put COT crates in
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 15:40:42.735 2003 -->
CHA source runs for wedges 0,1,2,21West, 0,2,3,5East.
<!-- Author: Andrea Sansoni :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171611-171770' target=_top>171611-171770</a>) -->


<!-- Date: Wed Oct 22 15:45:49.528 2003 -->
Experts requested CCAL,PCAL and WCAL crates, these have been released.  COT crates have been included. b0xft00 still gives errors and is ecluded. 
<!-- Author: Helen -->

<!-- Date: Wed Oct 22 15:48:19.566 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171776">171776</A>
ACTIVATE: L2_TORTURE[13,282,370]: CCAL,PCAL,WCAL,b0xtrp00 and b0xft00 excluded.
<!-- Author: Helen x2080 -->


<!-- Date: Wed Oct 22 15:49:53.305 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171777">171777</A>
ACTIVATE: QIE calorimeter calibration
<!-- Author: spezziga -->



<!-- Date: Wed Oct 22 15:52:47.566 2003 -->
CMP: Bottom, we've determined a problem in St32L1 (dead) to be the  
signal-out cable. 
Shift: Can you please take a cosmic run overnight,(I guess you  
already do that normally)  we need to check the whole CMP-Bot.  
Thank you.
<!-- Author: L. Cerrito and T. Vickey -->

<!-- Date: Wed Oct 22 15:56:59.220 2003 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>15:55:01</TD>                                                       
<TD ALIGN=CENTER>::</TD>                                                
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->


<!-- Date: Wed Oct 22 15:58:25.989 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171776">171776</A>
TERMINATE: end run
<!-- Author: Helen x2080 -->



<!-- Date: Wed Oct 22 16:00:58.283 2003 -->
<font color=red>Shift Summary: </font><pre>Another boring day shift.  
L2 fixed, CHA source test done, and doing L2_torture.</pre> 
<!-- Author: weiming -->

<!-- Date: Wed Oct 22 16:04:18.588 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=171777">171777</A>
TERMINATE: QIE calibration downloaded in all calorimeter crates
<!-- Author: spezziga -->