<!-- Title: Eve shift. Mon Apr 16, 2001 -->
<!-- SciCo: Steve Tether -->
<!-- DAQAce: Rob Snihur -->
<!-- MonAce: Eiko Yu -->
<!-- CO: Youngdo Oh -->
<!-- OpManager: Steve Hahn -->
<!-- Notes: -->
<pre></pre>


<!-- Date: Mon Apr 16 16:15:32 2001 -->
<pre>The VRB in  slot 12 of the b0eb16 crate was replaced 
with a spare with the latest version of non-SVX firmware. 
We will see if this makes the occaisional data corruption 
from this VRB go away.</pre>
<!-- Author: patrick -->


<!-- Date: Mon Apr 16 16:39:09 2001 -->
<pre>Ended the inherited (SEVB) run to add HEVB  & PCAL crates. 
Cleanup EVB. 
Partition failed with error: b0l3c16(Target failed), 
b0l3123 (Target failed), and more nodes same error. 
So I cleanup L3 farms, which takes a very long time (>7 min). 
Try to run again, but same error on partition. 
Page experts.</pre>
<!-- Author: rob -->


<!-- Date: Mon Apr 16 16:53:01 2001 -->
<pre>PHA EAST crate 8 wedge 4 gave an alarm. After 2 mins, it came  
back to normal. 
</pre>
<!-- Author: Eiko -->


<!-- Date: Mon Apr 16 16:54:35 2001 -->
<pre>Removed the PCAL crates, and dropped down to 4 L3 subfarms 
(previously I tried all subfarms).  Now the partition works, 
and we are up and running.</pre>
<!-- Author: rob :: (run <a href='/cgi-bin/runSummaryRun.cgi?112247' target=_top>112247</a>) -->


<!-- Date: Mon Apr 16 17:02:12 2001 -->
<pre>Talked to Jim Patrick.  PCAL cannot be the cause of partition 
failure, so I end the current run & add them back in. 
Using fewer L3 subfarms is probably what allowed us to run, 
so I will try to determine the offending subfarms, 
after verifying that adding back PCAL crates works OK. 
</pre>
<!-- Author: rob -->
<!-- Comment: Mon Apr 16 17;03;12 comment by...rob -->
Jim said the partition error may be due to a filter algorithm
not being on disk.

<!-- Comment: Mon Apr 16 17;06;04 comment by...rob -->
Running with PCAL crates and L3 subfarms 0,4,8,12 at 322 Hz.


<!-- Date: Mon Apr 16 17:33:23 2001 -->
L3 expert Boris came in and said I should not be using the 
last subfarm...Duh!  When I included all L3 subfarms except the 
last one, then it all works. 
Now running in calib continuous, as the trigger is being 
worked on until ~6pm.  
<!-- Author: rob :: (run <a href='/cgi-bin/runSummaryRun.cgi?112251' target=_top>112251</a>) -->


<!-- Date: Mon Apr 16 17:50:54 2001 -->
<pre>CEM, CHA and WHA HV lost connection to the local PC twice, 
but came back to normal within 1 min.</pre>
<!-- Author: Eiko -->


<!-- Date: Mon Apr 16 18:23:34 2001 -->
<pre>Investigated a problem with data taking with pulsing on CMU.  
The time delay between calib enable and L1A was off. Returned 
b0ccal14 and cmu01 back to ace. </pre>
<!-- Author: hyunsoo -->


<!-- Date: Mon Apr 16 18:29:04 2001 -->
<pre>Hyunsoo turned off CMP HV for the supervised access tomorrow 
(7AM to 10PM) for he might not be here tomorrow morning. </pre>
<!-- Author: Eiko -->


<!-- Date: Mon Apr 16 19:25:01 2001 -->
<pre>E:SVRAD2 and E:SVRAD3 stay at 28.5 Rads 
E:SVRAD0 and E:SVRAD1 stay at 0.95 Rads</pre>
<!-- Author: Eiko -->


<!-- Date: Mon Apr 16 19:59:14 2001 -->
Played with various run configurations. 
Grabbed all crates that I could. 
The following are not included: xtrp, musc, silicon, L2. 
When I tried to include MUSC00, coldstart failed with error: 
Resetting card in slot 8,9,13,14,15 failed. 
I logged into the crate & these cards do not show up in the map 
(the map shows cards only ion slots 3,4,6,7). 
Resetting the crate does not bring the offending cards back. 
Therefore, remove the crate.
<!-- Author: rob :: (run <a href='/cgi-bin/runSummaryRun.cgi?112259' target=_top>112259</a>) -->


<!-- Date: Mon Apr 16 20:16:29 2001 -->
<pre>Power supply monitor lost heart beats from 1RR21E-1. 
It's one of the b0vrb01 or b0vrb02 crates. 
After 10 mins, its heart beats came back. 
 
And now CEM, CHA, WHA GAM HV lost heart beats again. 
Back to normal after 2 mins.</pre>
<!-- Author: Eiko -->


<!-- Date: Mon Apr 16 20:40:07 2001 -->
<pre>Testing L2 processors...using table L2_RATE_TEST[1,101], 
with no silicon crates.  Included L2 Cal crates and decision 00. 
Chose L2 Mode = L2 processors.  After 178 events, we get L2 errors: Alpha in slot 3 is no longer processing at spot 6. 
I then logged into b0dap30, and ran the script ~jsm/alpha/alpha_check.  The output follows: 
 
Error: Alpha is no longer processing at spot 6: Ready to read Data 
 TSI handshake bits are 4fff0000 
 TSI startload bits are 1010184 
 TSI MODDONE bits are 4007ffff 
 Number of words for TL2D, buffer 0: 8b000000 
 Number of words for TL2D, buffer 1: 8b000000 
 Number of words for TL2D, buffer 2: ffff 
 Number of words for TL2D, buffer 3: 0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 0 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 1 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 2 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 3 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 4 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 5 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 6 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 7 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 8 is  0 
FISION: Input/output error (0x5): VME BERR received! 
 Address for DMA buffer 9 is  0                         
 
I then looked at the front panel of tsi00 and saw it was in state 
3 2 0.  This is not state 14, so no need to log into tsi00 and read some registers (even tho I did, I will not record the results here). 
 
Then uissued a HRR.  The error msgs keep coming in, and no events 
are flowing thru the system.  I ran the script again.  Only the 1st few lines of output are different: 
 
Error: Alpha is no longer processing at spot 6: Ready to read Data 
 TSI handshake bits are 4fff0000 
 TSI startload bits are 10101c0 
 TSI MODDONE bits are 4007f77f                                                                                           
 
I then tried a HRR again.  No effect. 
End the run & warm start. 
 
                                                                 </pre>
<!-- Author: rob :: (run <a href='/cgi-bin/runSummaryRun.cgi?112261' target=_top>112261</a>) -->


<!-- Date: Mon Apr 16 20:49:49 2001 -->
<pre>After a warm start, I get the same error after 855 events. 
b0tsi00 is in state 3 2 0, just as in last run. 
I ran the alpha_check script, and the 1st few lines of output are: 
 
Error: Alpha is no longer processing at spot 6: Ready to read Data 
 TSI handshake bits are 4fff0000 
 TSI startload bits are 1010184 
 TSI MODDONE bits are 4007ffff 
 Number of words for TL2D, buffer 0: 87000000 
 Number of words for TL2D, buffer 1: 87000000 
 Number of words for TL2D, buffer 2: ffff 
 Number of words for TL2D, buffer 3: 0         
 
Tried a HRR, but errors keep coming. 
Run script again, with output: 
 
                                                                Error: Alpha is no longer processing at spot 6: Ready to read Data 
 TSI handshake bits are 4fff0000 
 TSI startload bits are 10101c0 
 TSI MODDONE bits are 4007f77f 
 Number of words for TL2D, buffer 0: 87000000 
 Number of words for TL2D, buffer 1: 87000000 
 Number of words for TL2D, buffer 2: ffff 
 Number of words for TL2D, buffer 3: 0             
 
                                                
HRR one more time does not help, therefore end the run. 
                       </pre>
<!-- Author: rob :: (run <a href='/cgi-bin/runSummaryRun.cgi?112261' target=_top>112261</a>) -->


<!-- Date: Mon Apr 16 20:52:46 2001 -->
<pre>CHA CEM WHA lost connection to local PC and gave 
global alarms, I could see the colour turned pink 
in the global alarms window, but not in the HV 
summary. While eariler when the colour of HB turned 
pink in HV summary window, there's no global alarm. 
 
 </pre>
<!-- Author: Eiko -->


<!-- Date: Mon Apr 16 21:07:56 2001 -->
This time tried a cold start. However, b0l2cl03 does not 
finish the cold start transition.  I am able to ping this crate 
and log in.  Abort+reset and try again.  Now this crate is holding up the partition transition.  Reboot it with VxWorks.
<!-- Author: rob -->


<!-- Date: Mon Apr 16 21:16:05 2001 -->
Same L2 alpha processor error after 696 events.  
<!-- Author: rob :: (run <a href='/cgi-bin/runSummaryRun.cgi?112263' target=_top>112263</a>) -->


<!-- Date: Mon Apr 16 21:45:59 2001 -->
<pre>I changed the L2 executable for the L2_RATE_TEST back to 
the one used this weekend. We will see if that removes 
the "Alpha no longer processing errors". 
 
First see if adding the silicon causes the state 14 errors 
to show up. Then remove silicon and see if a reasonable 
20-50K run can be taken with L2_RATE_TEST without 
the "Alpha no longer processing errors" 
 
If the "no longer processing" error shows up,  
run the alpha_check program and record the output. 
Then try using the new run control feature of just coldstarting 
the l2de00 crate. Follow that by an HRR. 
Note that  coldstart can be very quick for the l2de00 crate, 
but in fact the Alpha board takes about 30seconds to reboot 
and load the L2 executable. One should wait that long 
before doing HRR. Also, ignore any "Alpha no longer 
processing errors" that show up during that 30seconds. 
 
</pre>
<!-- Author: Stephen Miller -->
<!-- Comment: Mon Apr 16 22;49;46 comment by...Eiko -->
<pre>What does 'just coldstarting the l2de00 crate' mean?

 </pre>
<!-- Date: Mon Apr 16 21:53:44 2001 -->
Could not get the following crates to coldstart: fib01, fib02, fib06. 
Logged into them and reset them with VISIONdemo.  
<!-- Author: rob -->
<!-- Comment: Tue Apr 17 07;11;50 comment by...SCN -->
<pre>Resetting the crates is 100% pointless...The only reason
these guys don't start is because the power is not on....</pre>
<!-- Date: Mon Apr 16 22:40:52 2001 -->
The silicon is not powered up!  Paged experts, who believed 
it should have been left at standby.  Expert now on the way in. 
Meanwhile, rc/java has run out of memory. 
Exited rc & started again.  
<!-- Author: rob -->



<!-- Date: Mon Apr 16 22:50:39 2001 -->
Removed the silicon crates, and now we are running 
L2_RATE_TEST[1,101] without any Alpha errors, 
because the executable has been changed to an earlier version. 
TS rate is about 20 Hz.
<!-- Author: rob :: (run <a href='/cgi-bin/runSummaryRun.cgi?112267' target=_top>112267</a>) -->
<!-- Comment: Mon Apr 16 23;02;55 comment by...rob -->
Most of the deadtime is due to L2, but I saw the rate increase
to ~50 Hz, for some unknown reason...

<!-- Comment: Mon Apr 16 23;05;47 comment by...rob -->
TS rate is back to 23 Hz.  The higher rate was observed for only ~2 min.

<!-- Comment: Mon Apr 16 23;37;57 comment by...Stephen Miller -->
<pre>If possible, please get a screen snapshot of the trigger
rates and deadtime from the trigger rate monitoring window.
Thanks.</pre>


<!-- Date: Mon Apr 16 23:53:20 2001 -->
SILICON: 
 
SVX B2W8 layer 0 trips frequently. Tomorrow we will replace  
the corresponding power supply which will hopefully cure that  
problem. For tonight's running you should check in case of  
readout errors from the silicon check the power supply GUI for  
tripps. In case of SB2W8, L0 trips please just take crate  
FIB_02 out of RC and keep going with the rest. In case of any  
other trip please take the corresponding crate out of the run  
in order to continue smoothly, page the silicon pager and make  
an entry iin the e-log!
<!-- Author: Petra -->


<!-- Date: Tue Apr 17 00:01:36 2001 -->
<font color=red>Shift Summary: </font><pre>
 We tried to get L2_RATE_TEST running both with and without silicon.

The first problem was getting L2 to run. We kept getting
"Alpha is no longer processing at spot 6" errors until
finally Steve Miller reverted the L2 executable to an earlier
version. That runs OK without Si.

Then late in the shift we found that Si HV had not been turned on. Petra came in to do it. Eventually we did run L2 with Si only
to get hung in state 14 right away.

Randy Herber has made a new version of his database browser.
See http://cdfdbb.fnal.gov:8520/cdfr2/databases 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   112247-112267
Delivered Luminosity   0  pb-1
Acquired Luminosity    0  pb-1
Efficiency             0
   
</td></tr></table></pre>
<!-- Author: Steve Tether -->


<!-- Date: Tue Apr 17 00:02:01 2001 -->
<pre>Silicon power expert arrives & powers it up. 
We start running with silicon crates & immediately (0 events) 
get into the state 14.  Ran the chack_alpha script, here's the output: 
 
Info: Alpha is okay. Currently at spot 7: Started Handshake with TSI 
 TSI handshake bits are 4fff0006 
 TSI startload bits are 10101a4 
 TSI MODDONE bits are 4007ffff 
 Number of words for TL2D, buffer 0: 8b000000 
 Number of words for TL2D, buffer 1: 8b000000 
 Number of words for TL2D, buffer 2: 87000000 
 Number of words for TL2D, buffer 3: 87000000     
 
 
Then logged into tsi00 & followed Stephen's instructions. 
The registers read: 
 
                  Offset? [00003010] (hex):  300c 
                 [00000006]  Change to: 
   Offset? [0000300C] (hex):  3010 
                 [00000008]  Change to:        
 
Did an HRR with not much effect. 
Got the state 14 again after no events. 
Did the same thing again. 
 
Info: Alpha is okay. Currently at spot 7: Started Handshake with TSI 
 TSI handshake bits are 4fff000a 
 TSI startload bits are 10101b4 
 TSI MODDONE bits are 4007ffff 
 Number of words for TL2D, buffer 0: 8f000000 
 Number of words for TL2D, buffer 1: 8f000000 
 Number of words for TL2D, buffer 2: 87000000 
 Number of words for TL2D, buffer 3: 87000000          
 
 
                       VISIONdemo-> 4 
To abort an operation, use a period (.) 
Single read/write operations on ts0 
   Offset? [00003010] (hex):  300c 
                 [0000000A]  Change to: 
   Offset? [0000300C] (hex):  3010 
                 [00000008]  Change to: 
   Offset? [00003010] (hex):  .                                                                                                                                         
                                                                                                                                  </pre> <font color=red><cite>(entry outside this shift's time range )</cite></font>
<!-- Author: rob -->