<!-- Title: Eve shift. Wed Feb 23, 2005 -->
<!-- SciCo: Jay Hauser -->
<!-- DAQAce: Xavier Bueso -->
<!-- MonAce: Elliot Lipeles -->
<!-- CO: Woo-Hyun Chung -->
<!-- OpManager: Steve Hahn -->
<!-- Notes: -->
<pre>Waiting for database computer fix and store.  Stack is pretty huge.</pre>


<!-- Date: Wed Feb 23 16:22:39.718 2005 -->
<pre>When using the new VRB we still saw the errors from svt06. 
 
Errors looked like: 
 
(MLE) b0svt06:Messenger:12:03:11 PM->FER_E_SVT, GB 6.1 nw=29 (mod7=1) ee=686718 event=10244  
(MLE) b0svt06:Messenger:12:03:17 PM->FER_E_SVT, GB 6.1 nw=29 (mod7=1) ee=70671e event=11708  
 
I put back the original VRB in slot 12. 
 
We also see byte shift errors coming from svt08. This shows 
up as TracerEventId errors from b0eb11 slot 9. 
 
This type of error occurs because of backplane noise in the  
front end crate. We saw the same type of error with the  
Rev D TDCs and modifications were done to the TRACER to filter 
the noise. 
 
There is already a modified TRACER in that crate. 
 
At some point we can try a different modified TRACER in that  
crate but I would like to do this test between stores when there 
is sufficient time. 
</pre>
<!-- Author: Frank Chlebana -->


<!-- Date: Wed Feb 23 16:55:26.893 2005 -->
<pre>VRB in slot 10 of b0eb11 giving data corruption errors --  
will try reseating it... 
 
</pre>
<!-- Author: W.Badgett -->
<!-- Comment: Wed Feb 23 17;03;15 comment by...WB -->
no luck -- same errors;  will narrow it down to crate

<!-- Date: Wed Feb 23 16:58:58.045 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194373">194373</A>
ACTIVATE: RunSet: CALIB_RP_SMQIE 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 16:59:35.200 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194373">194373</A>
TERMINATE: RunSet: CALIB_RP_SMQIE ENDED
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 17:02:00.824 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194374">194374</A>
ACTIVATE: RunSet: ACE_TOF_QIE_NEW 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 17:02:01.968 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194374">194374</A>
TERMINATE: RunSet: ACE_TOF_QIE_NEW ended
<!-- Author: Xavier 0x2080 -->



<!-- Date: Wed Feb 23 17:05:56.260 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194375">194375</A>
ACTIVATE: RunSet: ACE_LED_ALL 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 17:07:38.990 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194375">194375</A>
TERMINATE: RunSet: ACE_LED_ALL ended
<!-- Author: Xavier 0x2080 -->



<!-- Date: Wed Feb 23 17:12:55.176 2005 -->
<pre>Looking at b0eb11, slot 10 errors in details: 
 
Running vrbtest with 100000 passes, 512 extra bytes, random  
pattern, full data check: 
 
With all crates in the test: 
  b0cot00, 01, 02, 03, b0cmp00, b0musc00, b0cmx00, 01 
The test fails quickly, after 100 to 10000 events or so. 
 
But removing either of the following two crates: 
  b0cot00 
  b0cot03 
alone, results in the test passing. Argh. 
Perhaps this indicates some sort of cross-talk  
problem within the VRB when mutliple adjacent  
channels are pulsing data? 
 
</pre>
<!-- Author: W.Badgett -->

<!-- Date: Wed Feb 23 17:17:33.061 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194376">194376</A>
ACTIVATE: RunSet: XEF_ALL 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 17:19:58.015 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194376">194376</A>
TERMINATE: RunSET:SEF_ALL ended
<!-- Author: Xavier 0x2080 -->



<!-- Date: Wed Feb 23 17:42:47.036 2005 -->
<pre>Reseated the aux card (fibre receivers) in the  
back of the b0eb11 crate to see what would happen. 
The first test for b0eb11, slot 10 passed OK;  the  
second test included all three VRBs (slot 10,12,14)  
and passed OK.   The 3rd test with the latter configuration  
failed after 62 events with data corruption error  
reported.    Wah, wah, wah.   Too bad. 
 
Anyhow, let's try a L2 torture run to see how  
things look like. 
</pre>
<!-- Author: W.Badgett -->


<!-- Date: Wed Feb 23 17:50:35.435 2005 -->
<pre>We've been going thru agony trying to get L3+EVB  
to partition;  after numerous clean ups and retries,  
we still get: 
 
MLE) b0l3pcom2.fnal.gov:main:5:48:25 PM-&gt;Error on L3 node b0l3pcom2.fnal.gov (partition 0) Wed Feb 23 17:48:25 2005 l3proxy  
  in Initialize (L3ProxyPartition.cc:1013) 
  @L3_TRANS_FAIL - initialization timeout (will clean up):     b0l3c01  b0l3c02  b0l3c03  b0l3c04  b0l3c05  b0l3c06  b0l3c07  b0l3c08  b0l3c09  b0l3c10  b0l3c11  b0l3c13  b0l3c14  b0l3c15  b0l3c16 
 
Asked Jay to page EVB+L3 farm support. 
</pre>
<!-- Author: W.Badgett -->


<!-- Date: Wed Feb 23 18:14:17.250 2005 -->
When checking CEMLED calibration, there were error messages from DBANA and I couldn't get any result plots. Sent e-mail to experts.
<!-- Author: Woo-Hyun -->
<!-- Comment: Wed Feb 23 18;30;21 comment by...Woo-Hyun -->
We tried to bring up DBANA again when Willis looked.  DBANA doesn't  crash this time.  Hmmm...
<!-- Date: Wed Feb 23 18:21:48.689 2005 -->
Final protons being injected.
<!-- Author: hauser -->


<!-- Date: Wed Feb 23 18:28:31.161 2005 -->
<TABLE bgcolor=#cdc9a5> 
<TR><TH>Date</th><th>Time</th><th>BLM</th><th colspan=2>Dose</th></tr> 
<tr><td>2005.02.23</td><td>18:22:56</td><td>W Inner BLM</td><td>0.01</td><td>RADS</td></tr> 
<tr><td>2005.02.23</td><td>18:22:56</td><td>W Outer BLM</td><td>0.00</td><td>RADS</td></tr> 
<tr><td>2005.02.23</td><td>18:22:56</td><td>E Inner BLM</td><td>0.26</td><td>RADS</td></tr> 
<tr><td>2005.02.23</td><td>18:22:56</td><td>E Outer BLM</td><td>0.79</td><td>RADS</td></tr> 
</TABLE> 
<strong>Integrated dosage</strong>
<!-- Author: lipeles -->


<!-- Date: Wed Feb 23 18:28:59.999 2005 -->
Antiprotons being loaded.
<!-- Author: hauser -->



<!-- Date: Wed Feb 23 18:44:44.393 2005 -->
Pbars all loaded.
<!-- Author: Hauser -->

<!-- Date: Wed Feb 23 18:47:57.981 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194377">194377</A>
ACTIVATE: RunSet: AAA_SHOTSETUP  Trigger: PHYSICS_3_00 [2,566,536] 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 18:50:12.193 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194377">194377</A>
TERMINATE: start takinng data!!!
<!-- Author: Xavier 0x2080 -->



<!-- Date: Wed Feb 23 18:53:42.588 2005 -->
The problem with EVB+L3 looked somewhat fishy, but clear in the end. Cleanup EVB+L3 indicated no problems as both the farm and the SCPUs were fine. Given that I found the problem was in the scramnet ring... the scramnet module in the EVB crate where b0eb11 seats was not properly inserted. We re-inserted it, and all became fine. 
<br>
<!-- Author: Nuno/Ilya -->


<!-- Date: Wed Feb 23 19:04:53.660 2005 -->
MCR: scraping complete.
<!-- Author: hauser -->

<!-- Date: Wed Feb 23 19:11:57.849 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194378">194378</A>
 Activated at 2005.02.23 19:11:46
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:11:58.983 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194378">194378</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_HIGHLUM_3_00 [3,589,545] 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 19:16:43.489 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194378">194378</A>
 Terminated at 2005.02.23 19:16:35
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:17:11.757 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194378">194378</A>
TERMINATE: clean up  EVB

<!-- Author: Xavier 0x2080 -->
<!-- Comment: Wed Feb 23 19;24;30 comment by...Xavier -->
We got problems with b0cot07 and HRR didn't fix it.
The problem was a SM_SCPU_TIMEOUT which came after a SCPU_TRACER_EVENT_ID on b0eb11. Needed to end->reset->cleanup EVB. Worked fine
<!-- Comment: Wed Feb 23 19;29;17 comment by...Frank Chlebana -->
<pre>The first error that occurs looks like a byte shift error
coming from COT00.

SCPU-P0-E-TracerEventId: Event 34216, crate 85, channel 0 has either bad Tracer ID or bad markers around Tracer word.
 Tracer word event ID is 1, should be 8.
Tracer word = 0x00315555 (I think).
Channel data = 0x0000057d 0x00d80031 0x434f5444 0x00140001 0x00000001 0x0000055c 0x00000003 0x0000000f ...
... 0x0e000000 0x1a1d0006 0x0000039b 0x00000000 0xaaaa0000 0x5555aaaa 0x00315555 0x00000009 --&gt;  


After that there are a bunch of TracerEventId errors which 
look like the data gets out of sync - most likely as the 
result of the first error...  

Followed by the SCPU timeout

SM-P0-E-ScpuTimeout: SCPU ack timeout for Recover operation. --&gt;  
 
after which we have to clean up the EVB and restart a new
run.




</pre>
<!-- Date: Wed Feb 23 19:23:33.494 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3034&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3034' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3035&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3035' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3036&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3036' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3037&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3037' width='300'></a></center>
<!-- Author: lipeles -->
<!-- Comment: Wed Feb 23 19;24;10 comment by...lipeles -->
Shot+first hourly plots
<!-- Date: Wed Feb 23 19:23:37.882 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194379">194379</A>
 Activated at 2005.02.23 19:23:30
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:23:38.757 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194379">194379</A>
 Activated at 2005.02.23 19:23:33
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:23:47.274 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194379">194379</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_HIGHLUM_3_00 [3,589,545] 
<!-- Author: Xavier 0x2080 -->





<!-- Date: Wed Feb 23 19:25:51.065 2005 -->
<pre>CPR has already alarmed twice where it goes away immediately</pre>
<!-- Author: lipeles :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194379' target=_top>194379</a>) -->


<!-- Date: Wed Feb 23 19:26:20.336 2005 -->
<pre>In the previous run the initial SCPU error indicated  
the so-called "byte-shift" problem in b0eb11;  subsequent  
errors I cannot see an obvious problem, but I didn't look  
with a microscope. 
</pre>
<!-- Author: W.Badgett :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194378' target=_top>194378</a>) -->
<!-- Comment: Wed Feb 23 19;37;55 comment by...Frank Chlebana -->
The first error is due to the byte shift...

SCPU-P0-E-TracerEventId: Event 34216, crate 85, channel 0 has either bad Tracer ID or bad markers around Tracer word.
 Tracer word event ID is 1, should be 8.

Channel data = 0x0000057d 0x00d80031 0x434f5444 0x00140001 0x00000001 0x0000055c 0x00000003 0x0000000f ...
... 0x0e000000 0x1a1d0006 0x0000039b 0x00000000 0xaaaa0000 0x5555aaaa 0x00315555 0x00000009 -->  


We miss the fragment for Tracer ID 8

Probably miss event with Tracer ID 9

SCPU is looking for event with Tracer ID 9 but sees the 
next one...

SCPU-P0-E-TracerEventId: Event 34217, crate 49, channel 0 has either bad Tracer ID or bad markers around Tracer word.
 Tracer word event ID is 10, should be 9.
Tracer word = 0x00da0031 (I think).

Channel data = 0x000002cf 0x00da0031 0x434f5444 0x00140001 0x00000001 0x000002ae 0x00000003 0x0000000f ...
... 0x0005f0c8 0x039b1a20 0x00000000 0x00000000 0xaaaaaaaa 0x55555555 0x00da0031 0x80000000 -->  

<!-- Comment: Wed Feb 23 19;38;10 comment by...Frank Chlebana -->
<pre>The first error is due to the byte shift...

SCPU-P0-E-TracerEventId: Event 34216, crate 85, channel 0 has either bad Tracer ID or bad markers around Tracer word.
 Tracer word event ID is 1, should be 8.

Channel data = 0x0000057d 0x00d80031 0x434f5444 0x00140001 0x00000001 0x0000055c 0x00000003 0x0000000f ...
... 0x0e000000 0x1a1d0006 0x0000039b 0x00000000 0xaaaa0000 0x5555aaaa 0x00315555 0x00000009 --&gt;  


We miss the fragment for Tracer ID 8

Probably miss event with Tracer ID 9

SCPU is looking for event with Tracer ID 9 but sees the 
next one...

SCPU-P0-E-TracerEventId: Event 34217, crate 49, channel 0 has either bad Tracer ID or bad markers around Tracer word.
 Tracer word event ID is 10, should be 9.
Tracer word = 0x00da0031 (I think).

Channel data = 0x000002cf 0x00da0031 0x434f5444 0x00140001 0x00000001 0x000002ae 0x00000003 0x0000000f ...
... 0x0005f0c8 0x039b1a20 0x00000000 0x00000000 0xaaaaaaaa 0x55555555 0x00da0031 0x80000000 --&gt;  
</pre>
<!-- Date: Wed Feb 23 19:37:08.578 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194379">194379</A>
 Terminated at 2005.02.23 19:36:47
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:37:09.759 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194379">194379</A>
TERMINATE: Need to cleanup EVB
<!-- Author: Xavier 0x2080 -->
<!-- Comment: Wed Feb 23 19;40;42 comment by...Xavier -->
Now problem with b0cot09. Same problem than before. Cleanup successful. Now changing to the normal trigger table...

<!-- Comment: Wed Feb 23 19;41;37 comment by...Frank Chlebana -->
<pre>Same sequence of errors starting with a byte shift error in
COT00

SCPU-P0-E-TracerEventId: Event 264468, crate 0, channel 0 has either bad Tracer ID or bad markers around Tracer word.
 Tracer word event ID is 0, should be 4.
Tracer word = 0x00002500 (I think).
Channel data = 0x000006b3 0x00040031 0x434f5444 0x00140001 0x00000001 0x00000692 0x00000003 0x0000000f ...
... 0xb30003cf 0x0003a735 0x00000000 0xaa000000 0x55aaaaaa 0x31555555 0x00002500 0xd7800000 --&gt;  
</pre>

<!-- Comment: Wed Feb 23 19;45;05 comment by...Frank Chlebana -->
<pre>b0cot09 most likely the crate with the largest event size and
hence the one that returns busy.

The problem that starts it is the byte shift error from COT00</pre>

<!-- Date: Wed Feb 23 19:47:24.723 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194380">194380</A>
 Activated at 2005.02.23 19:47:04
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:47:25.708 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194380">194380</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [3,584,546] 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 19:50:29.160 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194380">194380</A>
 Terminated at 2005.02.23 19:50:13
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:50:52.807 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194380">194380</A>
TERMINATE: Cleanup eb
<!-- Author: Xavier 0x2080 -->
<!-- Comment: Wed Feb 23 19;54;42 comment by...Xavier -->
Same problem than before...

<!-- Comment: Wed Feb 23 19;55;53 comment by...Xavier -->
We have shephered b0cot00 which was the one causing problems, before starting new run (expert's advice)

<!-- Date: Wed Feb 23 19:57:24.780 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194381">194381</A>
 Activated at 2005.02.23 19:56:59
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:57:25.843 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194381">194381</A>
 Activated at 2005.02.23 19:57:02
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:57:26.998 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194381">194381</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [3,584,546] 
<!-- Author: Xavier 0x2080 -->


<!-- Date: Wed Feb 23 19:59:10.479 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194381">194381</A>
 Terminated at 2005.02.23 19:58:58
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 19:59:48.728 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194381">194381</A>
TERMINATE: cleanup EB
<!-- Author: Xavier 0x2080 -->
<!-- Comment: Wed Feb 23 20;00;47 comment by...Xavier -->
Expxert is going to power cycle COT00 which is the one causing all these problems...


<!-- Comment: Wed Feb 23 20;10;58 comment by...Frank Chlebana -->
<pre>We don't see the byte shift error...

But looks like the SCPU gets out of sync.

SCPU-P0-E-TracerEventId: Event 1173, crate 49, channel 0 has either bad Tracer ID or bad markers around Tracer word.
 Tracer word event ID is 6, should be 5.
Tracer word = 0x00360031 (I think).
Channel data = 0x0000024d 0x00350031 0x434f5444 0x00140001 0x00000001 0x0000022c 0x00000003 0x0000000f ...
... 0x00007e18 0x00000dc1 0x00000000 0x00000000 0xaaaaaaaa 0x55555555 0x00360031 0x80000000 --&gt;  
</pre>
<!-- Date: Wed Feb 23 20:02:15.620 2005 -->
<TABLE bgcolor=#add8e6> 
<TR><TH>Date</th><th>Time</th><th colspan=3>Quantity</th></tr> 
<tr><td>2005.02.23</td><td>19:57:57</td><td>TEVpr</td><td>8361.75</td><td>E09</td></tr> 
<tr><td>2005.02.23</td><td>19:57:57</td><td>TEVpb</td><td>1439.12</td><td>E09</td></tr> 
<tr><td>C:B0ILUM</td><td>100.43</td><td>E30</td><td></td><td>11800</td></tr> 
<tr><td>2005.02.23</td><td>19:58:13</td><td>C:B0LLUM</td><td>0.00</td><td>E30</td></tr> 
<tr><td>2005.02.23</td><td>19:58:13</td><td>C:B0PLOS</td><td>1316.40</td><td>Hz</td></tr> 
<tr><td>2005.02.23</td><td>19:58:13</td><td>C:B0ALOS</td><td>300.23</td><td>Hz</td></tr> 
</TABLE> 
<strong>Luminosity and losses</strong>
<!-- Author: lipeles -->

<!-- Date: Wed Feb 23 20:06:04.711 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382">194382</A>
 Activated at 2005.02.23 20:05:47
<!-- Author: RunControl -->


<!-- Date: Wed Feb 23 20:06:05.512 2005 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382">194382</A>
ACTIVATE: RunSet: AAA_CURRENT  Trigger: PHYSICS_3_00 [3,584,546] 
<!-- Author: Xavier 0x2080 -->



<!-- Date: Wed Feb 23 20:09:18.239 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3038&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3038' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3039&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3039' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3040&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3040' width='300'></a></center>
<!-- Author: lipeles -->
<!-- Comment: Wed Feb 23 20;10;29 comment by...lipeles -->
LOSTP is rising, it went from ~5k to ~7k in 50 minutes, but the
last 20 have been level.



<!-- Date: Wed Feb 23 20:11:03.790 2005 -->
Got a Done TO immediatly after starting on COT00. HRR didn't help. We shephered it.
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 20:11:51.492 2005 -->
Done TO on PCAL09. HRR fixed.
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 20:24:31.877 2005 -->
<pre>SCPU_BAD_CHANNEL_COUNTS in b0eb22. HRR fixed.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 20:32:38.829 2005 -->
<pre>The power cycle of crate b0cot00 (CONWT-3) seems to  
have ameliorated the so-called "byte-shift" problem  
on the front-end crate, which resulted in triggering  
an unrecoverable Tracer_Event_ID error in SCPU b0eb11. 
 
The current hypothesis as to why the errors were  
unrecoverable is related to the new code installed  
recently by Steve Tether to generate a VME bus-error  
when a SCPU VRB error is encountered.    Nuno has  
backed out the code to the previous version, which  
will be picked up at the next EVB cleanup.    
 
</pre>
<!-- Author: W.Badgett :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->
<!-- Comment: Wed Feb 23 20;53;09 comment by...Nuno -->
Okay, as requested above I've set things up in our scpu boot machine in order to switch to the standard scpu code version.
<!-- Date: Wed Feb 23 20:32:59.969 2005 -->
<pre>6 Si resonances in b0svx02. HRR fixed.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->
<!-- Comment: Wed Feb 23 20;42;00 comment by...MArcel -->
careful, it was only one, SL6 does not indicate the number
<!-- Date: Wed Feb 23 20:34:13.076 2005 -->
<pre>L2 decision TO</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->
<!-- Comment: Wed Feb 23 20;37;43 comment by...Xavier -->
Another one. HRR also fixed.


<!-- Comment: Wed Feb 23 20;45;19 comment by...Xavier -->
Another one.



<!-- Date: Wed Feb 23 20:58:23.647 2005 -->
<pre>Busy TO (b0svx06, b0cot05,b0cot17,b0svx02). Luckily HRR recovered.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 21:15:44.566 2005 -->
PSM alarm for 1RR18E_1 -5.2V channel was -6.22V. 
The next reading was normal.  
By the way this now causes an auto-halt in the RC, which 
may not really be neccessary.
<!-- Author: lipeles :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->
<!-- Comment: Wed Feb 23 21;50;48 comment by...Phil Schlabach -->
I wonder if it set off a trigger inhibit. We did recently have inhibits halt. Only trips need stop the run.
<!-- Comment: Wed Feb 23 22;15;56 comment by...Dale -->
Aye.  Via the run summary:<br>
<h3><a href="/java/cdfdb/servlet/Inhibits?RUN=194382">Trigger Inhibit</a> Events during Run <a href="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382">194382</a></h3><p>
</p><table>
</table><table border="1" cellpadding="4" cellspacing="2">
<tbody><tr><th>InhibitTime</th><th>AtEvent</th><th>RunTime<br><i>seconds from activate</i></th><th>Components</th></tr>
<tr><td><a href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Wed_Feb_23_2005_Eve&amp;anchor=20:06:04.711">2005.02.23 21:12:27</a></td>
<td align="right">1329304</td>

<td align="right">3998</td>
<td>VME Power:iFix</td>
</tr><tr>
</tr><tr><td><a href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Wed_Feb_23_2005_Eve&amp;anchor=20:06:04.711">2005.02.23 21:13:37</a></td>
<td align="right">1329304</td>
<td align="right">4067</td>
<td>null</td>
</tr><tr>
</tr></tbody></table>
<br>
More to the point if a power supply went off it would be the same as a large section of a detector going dead - halting in this situation is likely overkill but it gives the shift crew time to figure out the problem and is double insurance against taking data when we shouldn't.

<!-- Comment: Wed Feb 23 23;53;09 comment by...jdl -->
We occasionally get bad readings from the power supply monitor.  They do trigger an inhibit.  On the next reading 30-60 seconds later, when the voltage is read OK, the inhibit clears.  The bad reading problem is in my experience confined to the first floor and is probably some kind of noise pick-up in the monitor.  The halt is an insurance measure for the silicon people.  A power supply problem should stop the run.  That is why it is in the inhibit.  
<!-- Comment: Wed Feb 23 23;54;15 comment by...jdl -->
Also, I think the problem would fix itself if only we had the miniskirt inhibit logic working correctly.
<!-- Date: Wed Feb 23 21:18:36.963 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3041&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3041' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3042&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3042' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3043&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3043' width='300'></a></center>
<!-- Author: lipeles -->
<!-- Comment: Wed Feb 23 21;19;10 comment by...lipeles -->
LOSTP has leveled off and may be falling.


<!-- Date: Wed Feb 23 21:45:30.661 2005 -->
<pre>SCPU_BAD_CHANNEL_COUNTS in b0eb22. HRR fixed it.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->



<!-- Date: Wed Feb 23 22:00:29.833 2005 -->
CES noise plot shows choke on 11E seems to be workingLarr
<!-- Author: Larry  -->
<!-- Comment: Wed Feb 23 22;08;09 comment by...Phil -->
too think I said it was a sign of desparation, maybe I'll try it on mine

<!-- Comment: Wed Feb 23 22;11;18 comment by...ps -->
school marm says only one o in that too, oh no, my bad



<!-- Comment: Wed Feb 23 23;49;30 comment by...  -->
OOH NOO
<!-- Date: Wed Feb 23 22:18:55.129 2005 -->
<pre>busy TO in b0svx02. HRR recover.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 22:33:04.907 2005 -->
<pre>Done TO in b0cot09. HRR fixed.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 22:38:19.458 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3044&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3044' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3045&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3045' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3046&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3046' width='300'></a></center>
<!-- Author: lipeles -->
<!-- Comment: Wed Feb 23 22;38;42 comment by...lipeles -->
happy hourly plots


<!-- Date: Wed Feb 23 22:51:07.956 2005 -->
<pre>busy TO in b0svx06. HRR recovered.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 23:23:14.226 2005 -->
<pre>Done TO in b0cot08. HRR fixed.</pre>
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Wed Feb 23 23:37:48.880 2005 -->
<center><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3048&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3048' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3049&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3049' width='300'></a><a href='/cgi/elog/elog.pl?nb=2005&action=view&page=-3050&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/elog.pl?nb=2005&action=view&page=-3050' width='300'></a></center>
<!-- Author: lipeles -->
<!-- Comment: Wed Feb 23 23;41;54 comment by...lipeles -->
LDM1GC = longitudinal dampers adjusted...




<!-- Date: Wed Feb 23 23:56:14.994 2005 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194378>
194378</A> 
x2F74A</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_HIGHLUM_3_00 [3,589,545]</TD>                               
<TD ALIGN=CENTER>19:11:46</TD>                                          
<TD ALIGN=CENTER>19:16:35</TD>                                          
<TD ALIGN=CENTER>00:01:12</TD>                                          
<TD ALIGN=RIGHT>        920,979</TD>                                    
<TD ALIGN=RIGHT>        34,246</TD>                                     
<TD ALIGN=RIGHT>       6,703</TD>                                       
<TD ALIGN=RIGHT>    8.166</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>0</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194379>
194379</A> 
x2F74B</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_HIGHLUM_3_00 [3,589,545]</TD>                               
<TD ALIGN=CENTER>19:23:33</TD>                                          
<TD ALIGN=CENTER>19:36:47</TD>                                          
<TD ALIGN=CENTER>00:10:32</TD>                                          
<TD ALIGN=RIGHT>      7,121,540</TD>                                    
<TD ALIGN=RIGHT>       264,498</TD>                                     
<TD ALIGN=RIGHT>      53,717</TD>                                       
<TD ALIGN=RIGHT>   68.074</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194380>
194380</A> 
x2F74C</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [3,584,546]</TD>                                       
<TD ALIGN=CENTER>19:47:04</TD>                                          
<TD ALIGN=CENTER>19:50:13</TD>                                          
<TD ALIGN=CENTER>00:00:13</TD>                                          
<TD ALIGN=RIGHT>        473,643</TD>                                    
<TD ALIGN=RIGHT>         7,476</TD>                                     
<TD ALIGN=RIGHT>       1,569</TD>                                       
<TD ALIGN=RIGHT>    1.422</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD>                                                 
<TD ALIGN=CENTER>0</TD></TR>                                            
                                                                        
<TR><TD><A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382>
194382</A> 
x2F74E</TD>                                                             
<TD>BEAM</TD>                                                           
<TD>PHYSICS_3_00 [3,584,546]</TD>                                       
<TD ALIGN=CENTER>20:05:47</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>03:21:16</TD>                                          
<TD ALIGN=RIGHT>    274,204,239</TD>                                    
<TD ALIGN=RIGHT>     4,337,047</TD>                                     
<TD ALIGN=RIGHT>   1,121,469</TD>                                       
<TD ALIGN=RIGHT>  955.429</TD>                                          
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER></TD>                                                  
<TD ALIGN=CENTER>1</TD></TR>                                            
                                                                        
<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>23:55:03</TD>                                                       
<TD ALIGN=CENTER>03:33:14</TD>                                          
<TD ALIGN=RIGHT>     282,720,401</TD>                                   
<TD ALIGN=RIGHT>      4,643,267</TD>                                    
<TD ALIGN=RIGHT>     1,183,458</TD>                                     
<TD ALIGN=RIGHT> 1033.092</TD>                                          
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Wed Feb 23 23:56:47.744 2005 -->
Done TO b0cot12
<!-- Author: Xavier :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->


<!-- Date: Thu Feb 24 00:00:43.627 2005 -->
<font color=red>Shift Summary: </font><pre>Database computer fixed early during shift. 
 
18:21 protons injected. 
19:04 scraping was complete, initial luminosity was 118E30,  
a new record.  Losses were acceptable. 
 
However, DAQ problems were pretty bad. 
 
All runs below ended by DAQ failures related to b0cot00 crate: 
Run 194378 short run. 
Run 194379   "    " . 
Run 194380   "    " . 
Run 194381   "    " . 
 
Then problems mostly settle down: 
Run 194382 long run continues into owl shift. 
 
</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   194378-194382
Delivered Luminosity   1492.4  
Acquired Luminosity    1047.6  
Efficiency             70.2
   
</td></tr></table></pre>
<!-- Author: hauser -->


<!-- Date: Thu Feb 24 00:05:42.317 2005 -->
l2 decision timout. hrr fix
<!-- Author: efron :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=194382' target=_top>194382</a>) -->