<!-- Title: 19:26:41  Tue Apr  2 2002 -->
<!-- Crewchief: Joey -->
<!-- Op1: Johnny -->
<!-- Op2: Tommy -->
<!-- Op3: Dee Dee -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>Time for a new page</pre>


<!-- Date: Tue Apr 2 19:39:13 2002 -->
<pre>OKAY - now I've really changed the right DCAS. 
so, in crate b0l2cl02, 
slot 11 = original SN 36 
slot 9 = spare SN 68 with proper fram 
SN 46 taken from slot 9 marked as bad with yellow sticky in spare crate.  </pre>
<!-- Author: hray -->


<!-- Date: Wed Apr 3 06:29:35 2002 -->
<pre>I am running a SVTSPYMON test on b0dap65. It should not affect 
real SVTSPYMON, but in case ... Please let me know. - Subir</pre>
<!-- Author: Subir Sarkar -->


<!-- Date: Wed Apr 3 17:54:05 2002 -->
<pre>Summary of TTL arbitration tests: 
 
The errors of boards sending data has turned out to be TTL 
arbitration problems. Running with all 5 boards with 
L1 slot 10, Track slot 9, SVT slot 12, Clist slot 14, Isolist  
slot 20. Running L2_TORTURE at 20KHz L1A, we would get 
one TTL arbitration error every 10M events or so. The error 
would be reported by the alpha as the Tracklist board failing 
to send any MBUS words. The problem is that error on bossreq 
causes the tracklist board to get the bus just after it 
has passed on bossgrout to the board originally requesting 
the bus. Since the tracklist board is the first interface 
board it can get the bus. Most of the time, the Tracklist board 
would end up sending data at the same time as the L1 board. 
Rarely would it collide with Clist or SVT. 
Note that L1, Tracklist, SVT, Clist are all arbitrating on 
the bus. Isolist either sends nothing or sends later if it 
has clusters.  
 
If we ran with small delays (plus the delays 
due to tracking and clustering,) the actual overlap rate would 
be significantly smaller so the error rate would be quite low. 
If the errors can be caught by the alpha, and since they 
don't always cause sync errors, the L2 error trigger could  
simply flag the event.  
 
Several othe observations.: 
1. If the jumpers aren't in all the way to the end, then there 
are frequent MBUS collisions between the boards. The reason might 
be that the last board  is always driving  bossgrout high  
(unasserted), and with the jumpers in that is connected to boss. 
That allows boss to have a quicker rise time with less noise 
as it is turned off. Otherwise it is an open collector line being 
pulled high by termination resisotrs. 
 
2. If the L1 board is placed in the lowest slot, then there are 
frequent collisions between the two tracklist boards. Looking 
on the scope, the problem is similar to what was originally seen 
with the tracklist boards during initial testing. In that case 
boss wouldn't be asserted quickly enough, and noise would  
allow the 2nd board to get the bus. We had put stronger drivers 
on the boards, so boss is asserted much more quickly, however, 
it still takes about 20ns and there is a lot of noise on boss 
and bossreq. 
 
3. With no boards plugged in ,noticed that bossreq is held 
at 2.4V and boss is at 3V. I also measured their dc resistance 
(with power off) and found it to be 56 ohms for boss and bossreq 
vs about 130 ohms for other signals. Perhaps the termination 
should be changed for those two signals. 
 
Possible improvements: 
 - change tracklist firmware to enable drivers 50ns after 
boss is asserted, as turning on the drivers seems to be  
correlated with additional noise. 
 - Better noise protection in the TTL PAL. 
 - Modifying the termination for boss/bossreq. </pre>
<!-- Author: Stephen, Matt -->


<!-- Date: Thu Apr 4 13:07:57 2002 -->
<pre>TTL arbitration testing continued: 
 
 Summary: TTL arbitration works. We can run at high rate 
for order of 100 M events without errors. This is with all 
the interface boards sending data, and 2 reces boards  plugged 
in.   
 
We slightly changed the TTL arbitration pal firmware and this 
fixed the errors that would occur when the 2 track boards 
were not separated by another interface board. With the old 
firmware, the interface boards would collide very frequently. 
  
Another kind of error was when the tracklist and L1 board would 
collide. This was the error that would happen once every 10M 
events. In this case using a slower pal in the track board 
causes the error to go away. (The 2 speed grade pals are 15ns and 
7ns). The successful tests were with the new pal firmware but 
I suspect the old firmware plus the slower pal would also solve 
this error. 
 
 In one run of 80M events, one of the boards held on to the bus 
and prevented any board from sending data. HRR fixed this  
problem. This could have been an arbitration collision between 
the two tracklist boards.  In many other runs of order 20- 50M 
events, this error has not recurred. 
 
Many of these tests were with protons in the machine. With the 
L2_TORTURE table, this can give L1A rates of up to 200KHz, 
although the rates were all over the place. 
L2 would max out at about 32KHz. This table is dominated by 
low energy jet triggers, plus a base BC trigger with a L1 rate 
of 17kHz. 
 
 
</pre>
<!-- Author: Stephen, Matt, Greg -->
<!-- Comment: Thu Apr  4 16;55;27 comment by...Stephen -->
<pre>
We also ran with Alpha reading out Reces. In one mode it
would wait for the interface boards to finish sending data before
it would read the reces board (as is done currently in the top 
crate). A single reces read was done for every event and we never
 saw any errors.
 The code was then changed to force a single reces read right
after startload was asserted. In this case the alpha board
was arbitrating with the other interface boards. We would get
errors every few 100 events, but only from the L1 board. The
cause of this was clearly seen on the scope. The alpha firmware
turns off boss and dstrobe at the same time. However it still
takes 50-100ns for DDONE to go high. If the L1 board tried
to send immediately after the alpha, then it would see DDONE
as asserted (low) and wouldn't send dstrobe.
 Changing the alpha firmware to hold boss for 60-90ns after
dstrobe is released will solve this problem. </pre>
<!-- Date: Thu Apr 4 14:06:29 2002 -->
<p> 
It looks as if SB4W6 is the only wedge lost to SVT as a 
result of Saturday's 
<a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=105&frame=2&anchor=18:19:40'>accident</a>. 
SB4W6L2 lost an axial chip, and SB4W6L3 was already on the SVX 
<a href="http://nkek15.fnal.gov/cgi-bin/newproblemdb/DBreport.pl?detsys=SVX&wedge=B4W6&layer=L3">problem list</a> 
(for possibly recoverable readout problems). 
<p> 
SB0W7L2 lost an axial chip, but SVT has been using Layers 0134. 
<p> 
SB0W6L4, SB3WBL1, SB5WBL0, SB2W3L1 each lost a Z chip, which 
comes after all phi chips, so should be OK for SVT. 
<p>  
<!-- Author: Bill A -->



<!-- Date: Thu Apr 4 17:06:21 2002 -->
<pre>Remaining steps/issues for TTL arbitration 
 
 Goal is to run the bottom crate with beam and the standard 
physics table.  
 
1. We need a plan for getting the full complement of Reces boards 
in the bottom crate with fibers. We also noticed that the isolist 
board in the bottom crate had a high rate of errors. Those will 
need to be fixed. 
 
2. Repeat the tests without beam but with the final  
configuration. This includes 4 Reces boards plus L1 and 
track boards moved to slots 10 and 9. 
 
3. Take a shifts worth of beam, with 
the expectation that it would be physics quality data but 
that we would have freedom to adjust rates, have time to look 
at any errors that arise. 
 
4. Decide whether to turn bottom crate into the default or  
go back to the top crate.</pre>
<!-- Author: Stephen -->


<!-- Date: Thu Apr 4 21:08:27 2002 -->
The spy_manager exectuable currently running shall update a couple of ACNET test devices (Z:REMSE1 and Z:REMSE2) whith X and Y beam measurement the SVTbeam finder program.<br><br> 
 
In case of problems please revert to the usual spy_manager.  
(i.e. setup svtmon; cd ~cdf_svt/svttest/spymon ; spy_manager_Linux.exe >& spymanager.log & )
<!-- Author: Marco -->


<!-- Date: Fri Apr 5 14:00:37 2002 -->
I modified the front-end code so that both l2de crates will send 
out rate information through smartsockets. <P> 
Also the scaler monitor GUI can now optionally display the data 
from b0l2de01. When started from daqmon, there is now a popup 
menu allowing to select the l2 decision crate. From the command 
line, one can use "startMon ratemon b0l2de00" or "startMon ratemon b0l2de01". <P> 
None of the above has been tested thoroughly. 
  
<!-- Author: Arnd -->
<!-- Comment: Fri Apr  5 15;20;03 comment by...Arnd -->
Bill added L2-decision-autodetection; no need for any special
startup anymore, it will all be automagic.



<!-- Date: Fri Apr 5 17:05:29 2002 -->
FYI, Jonathan Lewis made this observation (see entry in the main log) after  installing the optical fibers:<BR> 
<BR> 
 
"Also, we found the iso-list 02 fiber appearing to be quite damaged. The jacket was pulled off. You may find problems  
with it."
<!-- Author: David -->


<!-- Date: Fri Apr 5 20:11:50 2002 -->
Testing the new cables in b0l2de00.<p> 
We found that ISOlist had errors consistent with not sending any data both in TrigMon and the alpha error checking.  We noticed that the ISOlist MASTER cable was not plugged into the jack in the splitter panel.  However, plugging the ISOlist MASTER cable in did not fix the problem.  We then tried moving the cables for the bottom crate to the ISOlist card in the top crate.  This also did not solve the problem.  Finally, we put the original ISOlist cables directly into ISOlist in the top crate and were able to take data.
<!-- Author: Matt and Masa -->


<!-- Date: Fri Apr 5 23:01:01 2002 -->
Moving to b0l2de01 for TTL tests.<p> 
We moved the original ISOlist fibers to the lower crate (leaving the new fibers to the upper and lower crate ISOlist cards disconnected and capped).  We also moved the 4 default Reces cards, and the Clist, XTRPlist, and SVTlist LVDS cables to the lower crate.<p> 
We had many problems running in the lower crate.  Initially, we found that Clist and SVTlist were not sending any data.  This was tracked down to a bad jumper in slot 11.  We went through a series of runs where DSTROBE was asserted for a long time and then the alpha would hang with a VME BERR.  The signature for this was seeing the tracklist boards' DSTROBE LEDs light up intensly, when neither board had its BOSS LED lit.  Before we could get a good picture on the scope, we saw that one of the Reces boards had no power.  We put a spare Reces in and the VME errors went away.  Masa also replaced the fuse on the Reces board that went bad, and it was fixed.  Then we ran briefly with L2 auto accepting and all of the data in TrigMon looked good.  However, as soon as we switched to L2 cutting again (L2_TORTURE_v-4) we had the runs consistenly stop because both Clist and L1 did not send.  We clearly saw this on the scope, with XTRPlist and SVTlist sending as normal.  We tried with L2_TORURE_v-4, PHYSICS_1_01_v-4 (both with Reces), and Stephen's special jsm_noreces_PHYSICS_1_01_v-4, and always had this error.  We tried reseating and recabling Clist with the cables to the top crate (which had worked previously), and reseating and replugging in L1 to no improvement.<p> 
For all runs the boards were in their normal slots as in the upper crate: XTRPlist 9, L1 10, SVTlist 12, Clist 14, Reces 15-18, ISOlist 20.  We copied the orig PHYSICS_1_01_v-4 back to the default, and put the upper crate back together with the original ISOlist fibers, and the new Clist fibers.  We couldn't find L2_TORURE_v-3, for which Stephen had made a special exe with no Reces and no L2 monitoring in RC.  Has it been retired?  
<!-- Author: Masa and Matt -->


<!-- Date: Sat Apr 6 22:01:39 2002 -->
<pre>Masa and I looked at some DCAS Isolation backplane signals in b0l2cl02.  We noticed that the suspect slot 9 already  
has 3 pins with jumpers,  we checked these but they 
seemed secure.   Since we were looking for stuck bits, it 
became clear we needed to repeat this without beam. </pre>
<!-- Author: Steve and Masa -->


<!-- Date: Sun Apr 7 10:54:27 2002 -->
<pre>Found that EM bit 3 had a new backplane problem in  
slot 9,  b0l2cl02.  Added another jumper (fourth one 
in this slot) and it seemed to fix the problem. Error 
rate should be back to 0.1% level. 
 
I took the TTL Isolist back to ANL for repair.  Will 
return Tuesday. </pre>
<!-- Author: Steve -->


<!-- Date: Sun Apr 7 16:02:32 2002 -->
<p>Did a test run with Stephen's latest buffer handling/reces readout fixes with the current trigger table  
 
<pre> 
PHYSICS_1_01[4,6] 
Tagset  Alphatag        L2runTag        l2codeTag       N 
6       v2_1_recesFix  v2_1_recesFix  v2_0_repoMar26    1 
</pre> 
See the <a href= 
'/cgi/elog/elog.pl?nb=2002&action=view&page=290&frame=2&anchor=15:00:00'> 
shift e-log</a>.</p> 
 
<p>The code worked as expected: 
<table cellpadding="0" cellspacing="1" border="0" width="90%"> 
     <tbody> 
       <tr> 
         <td valign="top">                      
      <table cellpadding="2" cellspacing="2" border="5" width="100%" 
 bgcolor="#ffffff"> 
           <tbody> 
             <tr align="center"> 
      <td valign="top" width="60%" bgcolor="#ffffff" colspan="2">L2 tagset 
V2<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top" width="60%">Empty current buffers:<br> 
      </td> 
      <td valign="top" align="center">  %<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top" width="60%">0<br> 
      </td> 
      <td valign="top">1.21<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">1<br> 
      </td> 
      <td valign="top">5.63<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">2<br> 
      </td> 
      <td valign="top">25.93<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">3<br> 
      </td> 
      <td valign="top">68.20<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">4<br> 
      </td> 
      <td valign="top">0.04</td> 
    </tr> 
    <tr> 
      <td valign="top" colspan="2">Deadtime 1.21% (28.57% L2)</td> 
    </tr> 
          
        </tbody>                      
      </table> 
         </td> 
         <td valign="top">                      
      <table cellpadding="2" cellspacing="2" border="5" width="100%" 
 bgcolor="#ffffff"> 
           <tbody> 
             <tr align="center"> 
               <td valign="top" width="60%" bgcolor="#ffffff" 
 colspan="2"> L2 tagset V6<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top" width="60%">Empty current buffers:<br> 
      </td> 
      <td valign="top" align="center">  %<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top" width="60%">0<br> 
      </td> 
      <td valign="top">0.72<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">1<br> 
      </td> 
      <td valign="top">2.46<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">2<br> 
      </td> 
      <td valign="top">10.05<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">3<br> 
      </td> 
      <td valign="top">31.87<br> 
      </td> 
    </tr> 
    <tr> 
      <td valign="top">4<br> 
      </td> 
      <td valign="top">54.94<br> 
    </td> 
    </tr> 
             <tr> 
      <td valign="top" colspan="2">Deadtime 0.65 (16.60% L2)</td> 
    </tr> 
                              
        </tbody>                      
      </table> 
         </td> 
       </tr> 
            
  </tbody>    
</table> 
   <br> 
Note the change in the fraction of time that four buffers are unused.  
 
This is the new default in the control room (PHYSICS_1_01[4,6]) and I will update the defaults for the gui, too.
<!-- Author: Peter Wittich -->
<!-- Comment: Sun Apr  7 16;08;51 comment by...Peter Wittich -->
This was at a lumi of 9.6E30. Run number of first new run with L2 tagset 6 is 142691.


<!-- Date: Tue Apr 9 16:57:43 2002 -->
<pre>Tried to re-establish operations with the lower crate. 
Used Stephen's new executable for L2_TORTURE_v-4. 
Saw lots of errors of this kind: 
 
Error: Clist data has buffer number 0 expected 2 for counter = 120              
Number of Clist MBUS words is 0                                                  
Error: Number of L1 words sent   is 0 for event 121 buffer 2 buffswitch 0        
Number of other words sent is 0 0 0                               L1 data is 800002 0 0                                             Error: Number of Track words sent   is 0 for event 121            Number of  words sent is 0 0 0 0                                  Error: Number of Svt words sent   is 0 for event 121              Error: Number of Cluster words sent   is 0       
 
The tracklist boards appear to not be sending any data at all. 
Will try to hook up a scope later tonight and look at the signals.                             </pre>
<!-- Author: natalia and greg -->


<!-- Date: Tue Apr 9 18:56:53 2002 -->
<pre>We pulled out all boards except L1 interface (and put on  
jumpers ... Seems to run fine with one board. 
Will reinsert one-at-a-time and see what hapens. 
</pre>
<!-- Author: Greg , Natalia -->
<!-- Comment: Tue Apr  9 19;00;08 comment by...Greg -->
<pre>run 1422796, SEVB
</pre>


<!-- Date: Tue Apr 9 19:12:31 2002 -->
<pre>Run 142797 -- put the two track list boards back (+ the L1 
interface board), but nothing else.  Everything works with this 
configuration.</pre>
<!-- Author: Natalia, Greg -->


<!-- Date: Tue Apr 9 19:25:52 2002 -->
<pre>Added CLIST to the two track list boards + L1 interface.  The errors are back.  This needs to be tested further tomorrow.</pre>
<!-- Author: Natalia, Greg -->


<!-- Date: Wed Apr 10 16:34:15 2002 -->
<pre>Tested new and inproved  fiber splitter for islist in 
bottom crate.  It did not work.</pre>
<!-- Author: Geg, Peter -->


<!-- Date: Wed Apr 10 16:47:14 2002 -->
<pre>Some good progress today in the bottom crate. 
CLIST now working!  The only thing we did 
was to jiggle the jumper on slot 13. Could  
this really be the cause/solution? 
---------------------------------------------- 
Running L2_TORTURE with all the good stuff. 
Cannot declare complete  success as the XTRP 
always sends BC=0, otherwise, there were no  
errors.</pre>
<!-- Author: Greg -->
<!-- Comment: Wed Apr 10 18;00;59 comment by...Greg -->
<pre>Clarification for David S.
Good stuff = all interface boards + HEVB.

</pre>


<!-- Date: Wed Apr 10 20:02:33 2002 -->
<pre>Matt stops by to help investigate XTRP issues. 
Offline we see XTRP data is all zero in TL2D.  We know 
the data cannot be identically zero (i.e. there 
must be something in the EoE or the  
XTRPlist would not send data to the alpha.) 
On the scope we see data entering and leaving the 
XTRPLIST board in the usual way. This means the   
buffer numbers are being passed correctly. 
The alpha thinks the XTRP BC  counter  is always zero 
and prints such an error to the minicom screen. 
------------------------------------------------------- 
We were using l2_torture[4,1,-] and saw the same errors  
in the bottom and top crate.  Here are 3 runs from the 
bottom crate 
142908 (extra1=1) 
142909 (extra1=1)  
142910 (extra1=0) 
---------------------------------------------------------- 
Our modest theory: Something is wrong with the L2 code. 
 
</pre>
<!-- Author: Greg and Matt -->
<!-- Comment: Thu Apr 11 09;02;32 comment by...Stephen -->
<pre>Don't be so modest. The code for L2_TORTURE [4,1] has now been
fixed.</pre>

<!-- Comment: Thu Apr 11 09;03;32 comment by...Stephen -->
<pre>Don't be so modest. The code for L2_TORTURE [4,1] has now been
fixed.</pre>


<!-- Date: Thu Apr 11 10:40:32 2002 -->
<pre>Run number 142936 with trigger table L2_torture[4,1,233} 
Running with the lower  crate. 
Here are the run parameters of interest: 
 
Dead time 2.6% (half due to L2) -- not clear why it's so high 
L2 latency is 40 microsec. 
L1 rate: 20 kHz 
L2 rate : 120 Hz 
L3 is not outputting anything in TrigMon 
 
Running for 960sec (100k events), see three CLIST errors (buffer number mismatch).  No other errors. 
 </pre>
<!-- Author: Jane, Greg, Natalia -->
<!-- Comment: Thu Apr 11 10;42;50 comment by...Stephen -->
<pre>The Clist sync errors are most likely pseudo-sync errors. That
is the buffer number coming with the cluster info is correct,
but the buffer number coming with the end of event word is
incorrect.  Since this is my old error checking code, the
buffer number from the eoe word is used. In Peter's official
error checking code, the buffer number with the clusters is used.</pre>


<!-- Date: Thu Apr 11 10:56:49 2002 -->
<pre>Run number 1429393: changed L1 prescale factor from 101 to 11. 
The dead time is 87%!! Mostly due to L2. 
</pre>
<!-- Author: Jane, Greg, Natalia -->


<!-- Date: Thu Apr 11 11:18:30 2002 -->
<pre>Greg is putting the two spare RECES boards, 06 and 07, into 
the lower crate slots 16 and 17, respectively.</pre>
<!-- Author: Jane, Greg, Natalia -->


<!-- Date: Thu Apr 11 11:21:48 2002 -->
<pre>TTL Isolist is still broken,  but clever enough to put 
all the failures in the overflow bin so it doesn't 
show up in the main Trigmon canvas.  The errors do show up 
in the "isolist_info_canvas".   It is a 40% error  
rate of overflows,  so this board cannot be used for 
physics runs.  I could take it back to ANL tomorrow for 
repair,  depending on the state of the TTL tests.</pre>
<!-- Author: Steve Kuhlmann  -->


<!-- Date: Thu Apr 11 12:53:10 2002 -->
<pre>Trying to run with Reces in bottom crate, special L2 exe which reads one reces board.  No Reces fibers connected yet, and spare Reces boards.  Run 142957 -- 62K events.  No errors in TrigMon.</pre>
<!-- Author: Jane, Greg -->


<!-- Date: Thu Apr 11 13:02:59 2002 -->
<pre>Karen moves real reces + fibers to bottom crate. 
Had to remove 4 more ccal crates due to pit activities. 
Starting run 142967</pre>
<!-- Author: Matt, Karen -->


<!-- Date: Thu Apr 11 13:37:09 2002 -->
<pre>142968: running with everything in the lower crate (4 reces boards). 
Running with L2_torture[4,1,233]. 
See these L1 errors at the run start-up (but not later): 
 
Error: Number of L1 words sent   is 0 for event 23 buffer 2 buffswitch 0 
Number of other words sent is 2 2 1  
L1 data is 2 0 0  
Error: Number of L1 words sent   is 0 for event 30 buffer 2 buffswitch 1 
Number of other words sent is 1 2 1  
L1 data is 2 0 0  
Error: Number of L1 words sent   is 0 for event 34 buffer 2 buffswitch 1 
Number of other words sent is 1 2 1  
L1 data is 2 0 0  
Error: Number of L1 words sent   is 0 for event 37 buffer 3 buffswitch 0 
Number of other words sent is 1 2 1  
L1 data is 2 0 0  
Error: Number of L1 words sent   is 0 for event 143 buffer 2 buffswitch 0 
Number of other words sent is 1 2 1  
L1 data is 2 0 0  
Error: Number of L1 words sent   is 0 for event 343 buffer 2 buffswitch 0 
Number of other words sent is 1 2 1  
L1 data is 2 0 0  
Error: Number of L1 words sent   is 0 for event 363 buffer 2 buffswitch 0 
Number of other words sent is 1 2 1  
L1 data is 2 0 0  
 
Also one CLIST error showed up again. 
There's ISOLIST errors in TrigMon. 
</pre>
<!-- Author: Matt, Greg, Natalia -->


<!-- Date: Thu Apr 11 13:42:58 2002 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-637&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-637' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-638&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-638' width='300'></a></center>
<!-- Author: Natalia -->
<!-- Comment: Thu Apr 11 13;44;57 comment by...Natalia and Matt -->
On the left are isolist errors.  On the right are the efficiencies for the Reces boards.  Otherwise all the data looks good in TrigMon for run 142968.  19kHz L1A, 81 Hz L2A, 1% deadtime.


<!-- Date: Thu Apr 11 13:58:26 2002 -->
<pre>Run 142968: it appears that the track list board and CLIST  
are colliding.  We are getting the following errors: 
 
Error: Clist data has buffer number 1 expected 0 for counter = 28943838  
Number of Clist MBUS words is 0  
Error: Number of Cluster words sent   is 0  
 Track bc 9 and svt bc 130 don't match 
Error: Number of Cluster words sent   is 0  
 Track bc 2 and svt bc 18 don't match 
Error: Number of Cluster words sent   is 0  
 Track bc 9 and svt bc 130 don't match 
Error: Number of Cluster words sent   is 0  
 Track bc 139 and svt bc 118 don't match 
Error: Number of Cluster words sent   is 0  
 Track bc 27 and svt bc 6 don't match 
Error: Number of Cluster words sent   is 0  
</pre>
<!-- Author: Matt, Natalia -->
<!-- Comment: Thu Apr 11 18;05;28 comment by...Natalia, Jane, Matt -->
Tried running this afternoon in various configurations to reproduce this error while triggering the scope on SVTlist and Clist raising their mod done lines within less than 500 nsec of each other.  However, once the calorimeter was given to Karen for calibrations, ISOlist to Steve for fixing, and EVB/L3 to Nahn for Si tests we were unable to get it back.
<!-- Date: Thu Apr 11 14:08:24 2002 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-640&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-640  width='300' align='top'></td><td>This plot shows missing bit erros from RECES.</a></td></tr></table>
<!-- Author: Natalia -->


<!-- Date: Thu Apr 11 16:40:47 2002 -->
<pre>Switched to PHYSICS_0_01[4,12,280], put L2 in auto accept mode because Steve K. took ISOlist for repairs.</pre>
<!-- Author: matt -->


<!-- Date: Thu Apr 11 16:43:10 2002 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-641&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-641  width='300' align='top'></td><td>XCES errors with the PHYSICS trigger table and 4 reces  cards in the lower crate.</a></td></tr></table>
<!-- Author: Matt -->


<!-- Date: Thu Apr 11 17:15:40 2002 -->
<pre>At Stephen's suggestion we took the new and improved 
firmware for RECES reading from the spare alpha (CDF D-2) 
and placed it in the TTL modified alpha. 
NOTE:  The prom from the TTL alpha was placed in the  
socket of the alpha called D-2 for safe keeping! 
</pre>
<!-- Author: Greg -->


<!-- Date: Thu Apr 11 17:41:53 2002 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-642&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-642' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-643&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-643' width='300'></a></center>
<!-- Author: Matt -->
<!-- Comment: Thu Apr 11 18;02;00 comment by...Masa and Matt -->
From the run with the better Reces firmware in the TTL modified alpha.  The reces trigger efficiencies look good, but the occupancies are strange.  Masa observed that this was the same problem with the showermax that was seen last sunday, and is caused by his code to send fake bits from the SMX through Reces.  In short, we think the new firmware fixed one problem, but testing it caused another problem in the SMX, which we fixed by taking another SMQIE calibration.



<!-- Date: Thu Apr 11 22:50:56 2002 -->
Still trying to find errors in the TTL crate:<p> 
Run 143065 with L2_TORTURE v4 has the full calorimeter and trigger.  Ran for 81M L1A at L1A rate 21.5 khz, L2A rate 100 hz with about 5% deadtime.  Saw 1 L1Mon error:<p> 
Error: Number of L1 words sent   is 0 for event 22 buffer 2 buffswitch 1<br>Number of other words sent is 1 1 1<br> L1 data is 2 0 0<p> 
and no other errors in TrigMon or the minicom with Debug flag set to 159, although the Reces plots still look strange.  This is with all boards but ISOlist.<p> 
Run 143068 with PHYSICS_0_01 v4 was with L2 tagging and ran with 99% deadtime and about 300 hz rates.  There were no errors in TrigMon (but strange Reces efficincies) or the minicom for 372k events.  Now I'm putting b0l2de00 back together.
<!-- Author: Matt -->
<!-- Comment: Thu Apr 11 22;52;05 comment by...Matt -->
The strange Reces plots look just like the one in <a href=
'/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=40&frame=2&anchor=13:42:58'>this</a> L2 e-log entry.


<!-- Date: Fri Apr 12 09:50:12 2002 -->
<pre>TTL Isolist is plugged back in,  cross your fingers...</pre>
<!-- Author: Steve Kuhlmann -->


<!-- Date: Fri Apr 12 17:26:13 2002 -->
<pre>Summary to date:  Reached new low with VME bus errors 
and no possibility to run.  Mass, Jane and Karen pulled 
entire crate apart and reseated the cards one by one 
including alpha, cpu, and tracer. 
Seemed to work better after all this using L2_torture 
but still track lists cards (at least)  would stop sending 
someties. 
--------------------------------------------------- 
Trying some more now with default physics table ...</pre>
<!-- Author: Greg,Masa,Matt -->


<!-- Date: Fri Apr 12 17:36:33 2002 -->
<pre>All boards in, default physics, SEVB. Run = 143199. 
Immediate and persistent L2 timeouts. Going to  
Auto Accpet mode so we can study. 
</pre>
<!-- Author: Greg -->


<!-- Date: Fri Apr 12 17:43:11 2002 -->
<pre>Run 14320. SEVB, AUTO accept. 
TriGmon shows L1 sending zero words sometimes. 
</pre>
<!-- Author: l2 crew -->


<!-- Date: Fri Apr 12 18:01:36 2002 -->
<pre>Boards not sending, etc. Trying L2 torture for the  
sake of it. 
 
</pre>
<!-- Author: Greg -->


<!-- Date: Fri Apr 12 18:13:45 2002 -->
<pre>Bottom crate still not working.  Matt notes  
there are still VME errors as FER does not  
read and report about the alpha in slot 3. 
We call Arnd who comes to  help out.</pre>
<!-- Author: Greg -->
<!-- Comment: Fri Apr 12 18;20;45 comment by...Greg -->
<pre>Reseated SVTlist and TRACKlist - gives successful
VME bus scans again.  Trying run 143204</pre>


<!-- Date: Fri Apr 12 18:39:26 2002 -->
<pre>All interface PALs upgraded to have latest TTL 
firmaware (MBuS 4-3 or MBUS 4-8 ARGO).  These 
have extra super glitch protection and the  
PALs are "slower" ... phone JSM for details. 
Note: L1 board reseated and recabled, SVT reseated to 
change the arbitration delay. 
--------------------------------------------------------- 
Delay status: L1 = none, SVT=2us, XTRP=1 us, CLIST=400us 
These are the settings as of right now.   
--------------------------------------------------------- 
</pre>
<!-- Author: Greg, Matt, Masa -->


<!-- Date: Fri Apr 12 18:46:26 2002 -->
<pre>Run 143206 - Claim is sometimes L1 interface board 
does not send data, otherwise run looks OK. 
Going to try running with only L1 interface and RECES. 
</pre>
<!-- Author: Greg -->


<!-- Date: Fri Apr 12 19:49:15 2002 -->
<pre>We have rediscovered Stephen's claim that the L1 board 
does not send sometimes for buffers 2 and 3.  To check 
this  we pulled all board out of the crate except L1. 
Upon reinserting the boards we discovered the VME 
problems had returned. REPEATED RESEATING of the  
cards made the VME problem go away. 
----------------------------------------------------------- 
In this run we have disabled the sending of PFRED  
data to the L1 card to see if that is the L1 problem. 
In addition, we have pulled a DCAS cable to fake clusters 
to check the ISOlist repair.</pre>
<!-- Author: Masa, Matt, Greg -->
<!-- Comment: Fri Apr 12 19;52;42 comment by...Greg -->
<pre>This run goes but no boards send any data ... Now it
looks like the TRACKlist boards are not sending.
</pre>

<!-- Comment: Fri Apr 12 19;55;20 comment by...greg -->
<pre>We stop  in defeat.</pre>


<!-- Date: Fri Apr 12 20:06:12 2002 -->
<pre>143212 - Back to top crate.  Checkout run before  
handing it over to the shift crew. 
 
</pre>
<!-- Author: Matt, Masa, Greg -->
<!-- Comment: Fri Apr 12 20;10;13 comment by...us -->
<pre>In this run we see the isolist buffer number 
mismatch.  Trying a cutting mode run to see 
if it crashes the run.</pre>


<!-- Date: Fri Apr 12 20:11:03 2002 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-644&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=l2&action=view&page=-644  width='300' align='top'></td><td>here is how it looks like</a></td></tr></table>
<!-- Author: matt -->


<!-- Date: Fri Apr 12 20:25:00 2002 -->
<pre>L2 buffer is always 3 in the simulated bits for both Clist and ISOlist.  The simulation comes from TC2D.  We took a run with L2 TORTURE in cutting mode and got no timeouts, so it seems that the comparison which the alpha error handling does for the L2 buffer is not finding errors.  This suggests that the problem is in the simulation somewhere.</pre>
<!-- Author: MMG -->
<!-- Comment: Fri Apr 12 20;54;07 comment by...Stephen -->
<pre>the L2_torture exe doesn't run the default error code, so
it will never produce L2 decision timeouts even in cutting mode.</pre>

<!-- Comment: Fri Apr 12 21;02;18 comment by...Matt -->
Steve K. responds to this plot that the L2 buffer is actually a comparison between TL2D and TSID (not TC2D), so that any run without L3 will have this effect for both Clist and ISOlist.  The title of this plot should be changed.

<!-- Comment: Fri Apr 12 21;35;47 comment by...hray -->
I had thought for the CLIST the buffer number comparison in the TrigMon plot really was the TC2D vs TL2D??



<!-- Date: Sun Apr 14 13:15:50 2002 -->
<pre>I swapped AMboard ser#07 out of svt05 slot 9. 
It intermittently failed AM write to one location. 
Ser#02 is now in and OK.</pre>
<!-- Author: Luciano -->


<!-- Date: Mon Apr 15 10:56:27 2002 -->
<pre>L2 Plan for today.  Move to bottom crate with  
everything but RECES and check out 
1) Isolist repair. 
2) L1 buffer problems 
3) General functioning. 
</pre>
<!-- Author: Greg -->
<!-- Comment: Mon Apr 15 10;57;37 comment by...Greg -->
<pre>Note:  We will be working with shift crew and beam ...
</pre>

<!-- Comment: Mon Apr 15 11;20;52 comment by...Greg -->
<pre>Beam just dumped. First run will be L2_torture table
with so fake xtrp or svt data.
</pre>


<!-- Date: Mon Apr 15 11:38:41 2002 -->
<pre>Run 143301 - No board is sending data 
------------------------------------------------- 
L1 data is 680485 80000000 0                                                    Error: Number of L1 words sent   is 0 for event 3439 buffer 3 buffswitch 1      Number of other words sent is 0 0 0                                             L1 data is 60040c 80000000 0                                                    Error: Number of L1 words sent   is 0 for event 3440 buffer 0 buffswitch 0      Number of other words sent is 0 0 0                                             L1 data is 680485 80000000 0                                                    Error: Number of L1 words sent   is 0 for event 3441 buffer 1 buffswitch 1      Number of other words sent is 0 0 0                                             L1 data is 60040c 80000000 0                                                    Error: Number of L1 words sent   is 0 for event 3442 buffer 2 buffswitch 0      Number of other words sent is 0 0 0                                             L1 data is 680485 80000000 0                                                    Error: Number of L1 words sent   is 0 for event 3443 buffer 3 buffswitch 1      Number of other words sent is 0 0 0                                             L1 data is 60040c 80000000 0                                                    Error: Number of L1 words sent   is 0 for event 3444 buffer 0 buffswitch 0      Number of other words sent is 0 0 0                                             L1 data is 680485 80000000 0                                                    Error: Number of L1 words sent   is 0 for event 3445 buffer 1 buffswitch 1      Number of other words sent is 0 0 0                                             L1 data is 60040c 80000000 0                                                   </pre>
<!-- Author: Greg -->
<!-- Comment: Mon Apr 15 11;39;34 comment by...Greg -->
<pre>TrigSim seems to show data being sent.
</pre>


<!-- Date: Mon Apr 15 11:47:21 2002 -->
<pre>Starting new run with Auto L2 Accept, hw=stephen4 and 
extra1=1.  Run 143302 
</pre>
<!-- Author: Greg -->


<!-- Date: Mon Apr 15 11:53:10 2002 -->
<pre>Run 134302 - No one sending.  I have checked jumpers, etc. 
Will ask Stepehen to look at code and data offline. 
</pre>
<!-- Author: Greg -->
<!-- Comment: Mon Apr 15 11;54;02 comment by...Greg -->
<pre>Here are some errors from the alpha
====================================
L1 data is eb8092a 60b4b4 ee0092a                                               Error: Number of L1 words sent   is 0 for event 28 buffer 3 buffswitch 1        Number of other words sent is 1 5 0                                             L1 data is 2 0 0                                                                Error: Number of L1 words sent   is 0 for event 29 buffer 0 buffswitch 0        Number of other words sent is 0 4 0                                             L1 data is ee0092a 401400 ee0092a                                               Error: Number of L1 words sent   is 0 for event 30 buffer 1 buffswitch 1        Number of other words sent is 0 5 0                                             L1 data is 2 0 0                                                                Error: Number of L1 words sent   is 0 for event 32 buffer 3 buffswitch 1        Number of other words sent is 0 4 0                                             L1 data is 2 0 0                                                                Error: Number of L1 words sent   is 2 for event 36 buffer 3 buffswitch 0        Number of other words sent is 1 1 2                                             L1 data is 2 0 0                                                                L1 data is eb8092a 60b4b4 ee0092a                                               Error: Number of L1 words sent   is 0 for event 38 buffer 1 buffswitch 1        Number of other words sent is 1 4 0                                             L1 data is eb8092a 3ca0 680421                                                  Error: Number of L1 words sent   is 0 for event 39 buffer 2 buffswitch 0        Number of other words sent is 0 4 0                                            
</pre>

<!-- Comment: Mon Apr 15 11;55;27 comment by...Greg -->
<pre>Here is the full "core dump" (159)
=============================
 Error: Clist data has buffer number 0 expected 3 for counter = 6787            Number of Clist MBUS words is 0                                                 Error: Number of L1 words sent   is 0 for event 6788 buffer 3 buffswitch 1      Number of other words sent is 0 0 0                                             L1 data is 2 0 0                                                                Error: Number of Cluster words sent   is 0                                      Error: Number of L1 words sent   is 0 for event 6789 buffer 0 buffswitch 0      Number of other words sent is 0 0 0                                             L1 data is 28 0 501003                                                          Error: Number of Cluster words sent   is 0                                     </pre>

<!-- Comment: Mon Apr 15 11;57;08 comment by...Greg -->
<pre>Seems to be problem only with CLIST and L1.
The front panel aritration lights on the 
TrackList card are flashing away!
</pre>


<!-- Date: Mon Apr 15 13:50:49 2002 -->
<pre>RUN 143303 
============ 
 Track bc 83 and svt bc 139 don't match                                          Track bc 15 and svt bc 0 don't match                                            Track bc 6 and svt bc 62 don't match                                            Track bc 139 and svt bc 3 don't match                                           Track bc 0 and svt bc 3 don't match                                             Track bc 62 and svt bc 106 don't match                                          Track bc 3 and svt bc 130 don't match                                           Track bc 3 and svt bc 24 don't match                                            Track bc 106 and svt bc 3 don't match                                           Track bc 130 and svt bc 77 don't match                                          Track bc 24 and svt bc 80 don't match                                           Track bc 3 and svt bc 133 don't match                                           Track bc 77 and svt bc 3 don't match                                            Track bc 80 and svt bc 27 don't match                                           Track bc 133 and svt bc 118 don't match                                         Track bc 3 and svt bc 115 don't match                                           Track bc 27 and svt bc 86 don't match                                           Track bc 118 and svt bc 77 don't match                                          Track bc 115 and svt bc 21 don't match                                          Track bc 86 and svt bc 3 don't match                                            Track bc 77 and svt bc 86 don't match                                           Track bc 21 and svt bc 77 don't match                                           Track bc 3 and svt bc 130 don't match                                           Track bc 86 and svt bc 121 don't match                                          Track bc 77 and svt bc 56 don't match                                           Track bc 130 and svt bc 18 don't match                                          Track bc 121 and svt bc 21 don't match                                          Track bc 56 and svt bc 3 don't match                                            Track bc 18 and svt bc 65 don't match                                           Track bc 21 and svt bc 18 don't match                                           Track bc 3 and svt bc 9 don't match                                             Track bc 65 and svt bc 21 don't match                                           Track bc 18 and svt bc 133 don't match                                          Track bc 9 and svt bc 115 don't match                                           Track bc 21 and svt bc 68 don't match                                           Track bc 133 and svt bc 0 don't match                                           Track bc 115 and svt bc 133 don't match                                         Track bc 68 and svt bc 127 don't match                                          Track bc 0 and svt bc 130 don't match                                         </pre>
<!-- Author: Greg -->


<!-- Date: Mon Apr 15 15:59:06 2002 -->
<pre>NEW RUN: L2_TORTURE, fake data, etc. 
All cards in bottom crate except reces. 
run number = 143313</pre>
<!-- Author: Greg -->
<!-- Comment: Mon Apr 15 16;16;05 comment by...Greg -->
<pre>Nevermind.  This run was usuing the top   crate.
</pre>


<!-- Date: Mon Apr 15 16:18:37 2002 -->
<pre>New run - 143314</pre>
<!-- Author: Greg -->
<!-- Comment: Mon Apr 15 16;23;04 comment by...Greg and Karen -->
<pre>Running! Set debug mask to 129, seeing lots of errors
in minicom where L1 does not send for buffer 2 or 3.
Going to check the scope.
</pre>

<!-- Comment: Mon Apr 15 17;04;40 comment by...Greg -->
<pre>In this run we cannot reproduce the L1 interface 
card not sending on buffer 2,3, however we do 
see occaisional errors, particularly when someone
grabs the bus and everyone stops sending.
</pre>

<!-- Comment: Mon Apr 15 17;18;02 comment by...Greg -->
<pre>To clarify above comment - Since we went back and got 
the scope hooked up we have not been able to get the
"desired" error.  Sometimes SVTlist doesn't send 
but then resumes sending ...</pre>


<!-- Date: Tue Apr 16 09:35:53 2002 -->
<pre>We begin today's torture.  I have setup the bottom crate with everything but RECES.  
The current status of the cards 
will be included in a comment below.  Running L2 torture 
with RECES readout removed.  Pulling 2 DCAS cables for  
clustering effects.  Fake tracks from XTRP and SVT.</pre>
<!-- Author: Greg -->
<!-- Comment: Tue Apr 16 09;39;55 comment by...Greg -->
<pre>All interface PALs upgraded to have latest TTL 
firmware (MBuS 4-3 or MBUS 4-8 ARGO).  These 
have extra super glitch protection and the  
PALs are "slower" 
--------------------------------------------------------- 
Delay status: L1 = none, SVT=2us, XTRP=1 us, CLIST=400us 
--------------------------------------------------------- 
</pre>


<!-- Date: Tue Apr 16 09:46:34 2002 -->
<pre>"Run  107892" going now.  NOTE: We are using the integration 
database so these data will not be available offline, 
however we can run consumers on them, etc.  Ending  
run to give up ccal :( 
</pre>
<!-- Author: Greg -->


<!-- Date: Tue Apr 16 09:52:32 2002 -->
<pre>Run 107893.  No ccal (but remember the 2 pulled DCAS cables.) 
Running alpha_debug_linux01 with full error mask. 
The only visible error is L1 not sending for buffers 0,1,2,3. 
Going to check cables and jumpers, etc. 
</pre>
<!-- Author: Greg -->
<!-- Comment: Tue Apr 16 10;02;27 comment by...Greg -->
<pre>Ooops!  Only the L1 errors were enabled.  With 159, we
see errors from everybody.  No one is sending data.
--------------------------------------------------------
 Error: Clist data has buffer number 0 expected 1 for 
counter = 13           
   Number of Clist MBUS words is 4                                                 Clist data 0 is eb8092a 2014a0                                                  Clist data 1 is eb8092a 1028a0                                                  Clist data 2 is eb8092a 3ca0                                                    Clist data 3 is 68041b 80000000                                                 Clist data 4 is 101 7fe                                                         Clist data 5 is 7fe07fe 7fe                                                     Clist data 6 is 141 7fe                                                         Clist data 7 is 7fe07fe 7fe                                                     Error: Number of L1 words sent   is 0 for event 14 buffer 1 buffswitch 1        Number of other words sent is 1 7 4                                             L1 data is cc230318 c1081181 d000948                                             Track bc 0 and svt bc 254 don't match                                                                                                                                                                                                           Error: Clist data has buffer number 0 expected 2 for counter = 14              Number of Clist MBUS words is 1                                                 Clist data 0 is eb8092a b078a0                                                  Clist data 1 is eb8092a a08ca0                                                  Error: Number of Track words sent   is 0 for event 15                           Number of  words sent is 1 0 7 1                                                 Track bc 0 and svt bc 83 don't match                                                                                                                                                                                                            Error: Clist data has buffer number 3 expected 0 for counter = 16              Number of Clist MBUS words is 4                                                 Clist data 0 is 2 0                                                             Clist data 1 is 0 0                                                             Clist data 2 is 7fe07fe 80400002                                                Clist data 3 is 7fe07fe 400002                                                  Clist data 4 is 7fe07fe 404002                                                  Clist data 5 is 7fe07fe 80404002                                                Clist data 6 is 7fe07fe 80800002                                                Clist data 7 is 7fe07fe 800002                                                   Track bc 254 and svt bc 0 don't match                                         </pre>
<!-- Date: Tue Apr 16 09:59:19 2002 -->
<pre>Jane arrives!!!</pre>
<!-- Author: Greg -->



<!-- Date: Tue Apr 16 10:13:43 2002 -->
<pre>So, the alpha error messages claim everyone is unhappy. 
Switching to alpha Auto Accept while Jane tries to  
get TrigMon going. 
</pre>
<!-- Author: Greg -->


<!-- Date: Tue Apr 16 10:14:24 2002 -->
<pre>Stephen arrives!!!  Now stuff will work??</pre>
<!-- Author: Greg -->


<!-- Date: Tue Apr 16 10:38:06 2002 -->
<pre>Natalia arrives!!!!! Now it will work ....</pre>
<!-- Author: Greg -->