<!-- Title: 07:59:52  Fri Jan 18 2002 -->
<!-- Crewchief: SiliFolks -->
<!-- Op1: &nbsp; -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>Yup</pre>


<!-- Date: Fri Jan 18 08:02:37 2002 -->
<pre>I went through and made a: 
DAVID ERROR MONITOR LADDER PROBLEM HISTORY 
 
------------------------------------------------------------ 
AVDD2 
------------------------------------------------------------ 
0 0 2	87	AVDD2 
0 1 2	??	AVDD2			(started last night) 
3 3 4	A2	DRO (AVDD2) 
4 3 2	A4	DRO (AVDD2) 
 
------------------------------------------------------------ 
'7F' (number is channel, or mobile if it moves around) 
------------------------------------------------------------ 
0 9 2	A2	7f 08			1%      until 21459  
1 0 4	A2	7f 4b			(runs 12128,29,d8) 
1 1 4	A2	7f 37			(runs 212d8 and 21451) 
2 8 1	83	7f 45				since 2111b 
3 1 1	85	7f 16			0.2	211c6 to 211f2 
3 6 2	86	7f 13			&lt;0.1 
3 7 4	A2	7f 17			0.5%	until 21459 
4 0 3	85	7f 41			rare	from 21321 
4 3 4	89	7f 1b			2% 
4 a 1	84	7f 11			1%	intermittent 
5 8 2	A3	7f 2D			0.1% 
0 4 0	83,82	7f mobile		1% 
0 b 3	89	7f mobile		1% 
1 6 0	83	7f mobile		&lt;0.1 
2 0 0	83	7f mobile		0.1 
2 1 4	8D	7f mobile		7%	since 2100e  
2 6 4	8D	7f mobile		2% 
3 3 3	A2	7f mobile		&lt;0.1 
3 a 4	all	7f mobile		rare 
3 b 3	89	7f mobile		0.2	 
5 2 2	89-A4	7f mobile (most in 85)	0.1	until 214f9 
4 2 4	8D	7f mobile		1% 
4 5 2	87	7f mobile		&lt;0.1	until 214f9 
4 8 4	many	7f mobile		rare 
4 a 2	89	7f mobile		0.3% 
5 1 0	83	7f mobile			until 1f17a 
5 1 1	85	7f mobile			until 1f178 
5 5 4	8D	7f mobile		rare 
5 7 4	8D	7f mobile			until 1e888 
 
------------------------------------------------------------ 
Other chip related bit 
------------------------------------------------------------ 
0 7 3	all	CapID bit 2		1%	since 1f178  
1 5 1	83	72-&gt;52 RA only		2% 
1 5 2	all	5x-&gt;7x			2% 
1 b 2	all	bit 7: 5x-&gt;dx, 7x-&gt;fx	&lt;0.1 
2 1 3   A3      bit 5 low		2% 
4 3 0	A1	CapID bit 3		0.3%	since 2147e 
4 3 3	A0	7F-&gt;5F NN only		1% 
4 6 2	85	bit 0 hi (32-&gt;33 RA)	0.5% 
4 7 1	all	Bit 2 hi NN only	3% 
 
------------------------------------------------------------ 
Chip ID problems 
------------------------------------------------------------ 
1 a 2	86	86-&gt;06 w/wrong CapID	10% 
2 1 1	84-&gt;80;83-&gt;01,03,81;a2-&gt;22;a1-&gt;21; CapID wrong for bit 7 chip errors 
4 2 0	Ax	Ax-&gt;Ex,CapID wrong	4% 
5 4 3		bit 4 hi (most A0-&gt;B0)	0.1 
 
------------------------------------------------------------ 
optical-like bit problems 
------------------------------------------------------------ 
1 5 4		bit 2 hi optical	90% 
2 0 4		bit 0 hi optical	3%	since 216f2 
2 5 0		bit 1 stuck low		100%	since 21664 
2 7 1		bit 1 stuck low		100% 
3 1 3		bit 0 hi optical	3%	since 2106f 
4 6 3		bit 3 stuck low 	0.1%	since 21451 
4 b 2		bit 1 low optical	3%	since 21355 
5 9 2		bit 1 stuck low		100%	 
5 b 0		bit 0 stuck high	100% 
5 b 1		bit 3 high optical	rare 
 
------------------------------------------------------------ 
Dropped RO/c2c2/obdv 
------------------------------------------------------------ 
0 3 4	A3	DRO (very avdd2ish)	100%	since last night 
0 5 1	85	DRO			100% 
1 2 2	89	severe 7f?, CapID bit 0	100% 
	85-A4	DRO     
3 2 0		DRO (no single chip)		since 21239 
4 0 4	8D	DRO immediate		100% 
4 5 4		DRO 8D mostly		100%	since 21062 
5 1 3	89	DRO			3%	until 1f178 
5 1 4	A6	DRO			100% 
5 6 2	85	DRO RA only		20%	 
5 6 3	A3,A0	DRO RA only		 
 
------------------------------------------------------------ 
Event fragments (looks like Frankenstein stitched together 2 
------------------------------------------------------------ 
2 0 1		event fragments		 
2 0 2		event fragments		 
2 6 2	A0	DRO, event fragments		since 21664 
2 7 4		event fragments			since 21664 
3 6 3		event fragments		rare	since 214f9 
4 8 2		DRO, event fragments	70%	 
 
------------------------------------------------------------ 
OBDV 
------------------------------------------------------------ 
0 2 *		c2c2 (when currents bad) 
0 3 2		c2c2			100%	since 21664  
1 4 0	83	00f3 after chip NN only 
1 4 *		00f3 after fibid	 
3 2 2-4		c2c2			&lt;2%	since 21173 
4 a 3	89	00f3 or DRO after chip	100%    since 217d1 
5 3 0		c2c2				21738, 21731 
 
------------------------------------------------------------ 
Other chip related 
------------------------------------------------------------ 
0 8 4	A6	out of sync		~once per long run 
3 6 4	A4	CapID lost 
5 3 4	89	CapID stuck at 1	100%	 
5 a 4	A1	duplicate chan NN only  50% 
</pre>
<!-- Author: Lester -->
<!-- Comment: Fri Jan 18 08;08;07 comment by...l -->
<pre>Also, there are a lot of low level bit 0 high in 
Bulkhead 0 south.
</pre>

<!-- Comment: Fri Jan 18 08;31;46 comment by...l -->
<PRE>A typical 'fragment' event:

Event Number: 10314
<font color=yellow><B>10</B></font><font color=yellow><B>50</B></font><font color=blue>0d</B></font><font color=cyan><B>0b</B></font> <font color=yellow><B>04</B></font><font color=yellow><B>10</B></font><font color=yellow><B>41</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> <font color=yellow><B>08</B></font><font color=yellow><B>16</B></font><font color=yellow><B>08</B></font><font color=yellow><B>16</B></font> <font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font><font color=yellow><B>00</B></font> 
<font color=black>e2</B></font><font color=yellow><B>74</B></font><font color=green>3b</B></font><font color=cyan>fc</B></font> <font color=green><B>8d</B></font><font color=black>26</B></font><font color=blue><B>00</B></font><font color=black>f3</B></font> <font color=blue><B>21</B></font><font color=black>f3</B></font><font color=blue><B>22</B></font><font color=black>05</B></font> <font color=blue><B>23</B></font><font color=black>f3</B></font><font color=blue><B>3f</B></font><font color=black>01</B></font> <font color=blue><B>40</B></font><font color=black>09</B></font><font color=blue><B>41</B></font><font color=black>01</B></font> <font color=blue><B>4c</B></font><font color=black>03</B></font><font color=blue><B>4d</B></font><font color=black>07</B></font> <font color=blue><B>4e</B></font><font color=black>f3</B></font><font color=blue><B>56</B></font><font color=black>f3</B></font> <font color=blue><B>57</B></font><font color=black>07</B></font><font color=blue><B>58</B></font><font color=black>03</B></font> <font color=green>0ns</font>
<font color=blue><B>59</B></font><font color=black>0f</B></font><font color=blue><B>5a</B></font><font color=black>04</B></font> <font color=blue><B>5f</B></font><font color=black>03</B></font><font color=blue><B>60</B></font><font color=black>05</B></font> <font color=blue><B>61</B></font><font color=black>f3</B></font><font color=blue><B>64</B></font><font color=black>04</B></font> <font color=blue><B>65</B></font><font color=black>08</B></font><font color=blue><B>66</B></font><font color=black>05</B></font> <font color=blue><B>69</B></font><font color=black>03</B></font><font color=blue><B>6a</B></font><font color=black>06</B></font> <font color=blue><B>6b</B></font><font color=black>03</B></font><font color=blue><B>6f</B></font><font color=black>02</B></font> <font color=blue><B>70</B></font><font color=black>07</B></font><font color=blue><B>71</B></font><font color=black>06</B></font> <font color=blue><B>72</B></font><font color=black>03</B></font><font color=blue><B>78</B></font><font color=black>02</B></font> 
<font color=blue><B>79</B></font><font color=black>06</B></font><font color=blue><B>7a</B></font><font color=black>01</B></font> <font color=blue><B>7b</B></font><font color=black>06</B></font><font color=blue><B>7c</B></font><font color=black>02</B></font> <font color=green><B>8c</B></font><font color=black>26</B></font><font color=blue><B>13</B></font><font color=black>02</B></font> <font color=blue><B>14</B></font><font color=black>0b</B></font><font color=blue><B>15</B></font><font color=black>f3</B></font> <font color=blue><B>5a</B></font><font color=black>03</B></font><font color=blue><B>5b</B></font><font color=black>05</B></font> <font color=blue><B>5c</B></font><font color=black>04</B></font><font color=green><B>8b</B></font><font color=black>26</B></font> <font color=green><B>8a</B></font><font color=black>26</B></font><font color=blue><B>1b</B></font><font color=black>f3</B></font> <font color=blue><B>1c</B></font><font color=black>05</B></font><font color=blue><B>1d</B></font><font color=black>f3</B></font> <font color=green>1360ns</font>
<font color=blue><B>5f</B></font><font color=black>f3</B></font><font color=blue><B>60</B></font><font color=black>05</B></font> <font color=blue><B>61</B></font><font color=black>f3</B></font><font color=green><B>89</B></font><font color=black>26</B></font> <font color=blue><B>0f</B></font><font color=black>f3</B></font><font color=blue><B>10</B></font><font color=black>05</B></font> <font color=blue><B>11</B></font><font color=black>f3</B></font><font color=blue><B>43</B></font><font color=black>01</B></font> <font color=blue><B>44</B></font><font color=black>05</B></font><font color=blue><B>45</B></font><font color=black>f3</B></font> <font color=red><b>19</B></font><font color=black>02</B></font><font color=blue><B>1a</B></font><font color=black>05</B></font> <font color=blue><B>1b</B></font><font color=black>01</B></font><font color=blue><B>7d</B></font><font color=black>f3</B></font> <font color=blue><B>7e</B></font><font color=black>05</B></font><font color=blue><B>7f</B></font><font color=black>f3</B></font> <font color=red>2240ns</font>
<font color=green><B>8a</B></font><font color=red><b>15</B></font><font color=blue><B>36</B></font><font color=black>01</B></font> <font color=blue><B>37</B></font><font color=black>05</B></font><font color=blue><B>38</B></font><font color=black>f3</B></font> <font color=green><B>89</B></font><font color=red><b>15</B></font><font color=blue><B>3c</B></font><font color=black>f3</B></font> <font color=blue><B>3d</B></font><font color=black>28</B></font><font color=blue><B>3e</B></font><font color=black>f3</B></font> <font color=green><B>88</B></font><font color=red><b>15</B></font><font color=blue><B>10</B></font><font color=black>f3</B></font> <font color=blue><B>11</B></font><font color=black>05</B></font><font color=blue><B>12</B></font><font color=black>f3</B></font> <font color=blue><B>69</B></font><font color=black>f3</B></font><font color=blue><B>6a</B></font><font color=black>06</B></font> <font color=blue><B>6b</B></font><font color=black>02</B></font><font color=green><B>87</B></font><font color=red><b>15</B></font> <font color=red>2500ns</font>
<font color=blue><B>18</B></font><font color=black>f3</B></font><font color=blue><B>19</B></font><font color=black>05</B></font> <font color=blue><B>1a</B></font><font color=black>f3</B></font><font color=blue><B>35</B></font><font color=black>03</B></font> <font color=blue><B>36</B></font><font color=black>0c</B></font><font color=blue><B>37</B></font><font color=black>02</B></font> <font color=blue><B>41</B></font><font color=black>f3</B></font><font color=blue><B>42</B></font><font color=black>06</B></font> <font color=blue><B>43</B></font><font color=black>03</B></font><font color=green><B>a6</B></font><font color=red><b>15</B></font> <font color=blue><B>04</B></font><font color=black>f3</B></font><font color=blue><B>05</B></font><font color=black>05</B></font> <font color=blue><B>06</B></font><font color=black>04</B></font><font color=blue><B>3d</B></font><font color=black>03</B></font> <font color=blue><B>3e</B></font><font color=black>06</B></font><font color=blue><B>3f</B></font><font color=black>02</B></font> <font color=red>3500ns</font>
<font color=blue><B>47</B></font><font color=black>01</B></font><font color=blue><B>48</B></font><font color=black>06</B></font> <font color=blue><B>49</B></font><font color=black>04</B></font><font color=blue><B>54</B></font><font color=black>f3</B></font> <font color=blue><B>55</B></font><font color=black>05</B></font><font color=blue><B>56</B></font><font color=black>f3</B></font> <font color=blue><B>5e</B></font><font color=black>f3</B></font><font color=blue><B>5f</B></font><font color=black>06</B></font> <font color=blue><B>60</B></font><font color=black>03</B></font><font color=blue><B>62</B></font><font color=black>03</B></font> <font color=blue><B>63</B></font><font color=black>05</B></font><font color=blue><B>64</B></font><font color=black>02</B></font> <font color=blue><B>6b</B></font><font color=black>f3</B></font><font color=blue><B>6c</B></font><font color=black>06</B></font> <font color=blue><B>6d</B></font><font color=black>f3</B></font><font color=blue><B>6e</B></font><font color=black>06</B></font> 
<font color=blue><B>6f</B></font><font color=black>03</B></font><font color=blue><B>71</B></font><font color=black>01</B></font> <font color=blue><B>72</B></font><font color=black>06</B></font><font color=blue><B>73</B></font><font color=black>f3</B></font> <font color=green><B>a5</B></font><font color=red><b>15</B></font><font color=blue><B>52</B></font><font color=black>f3</B></font> <font color=blue><B>53</B></font><font color=black>06</B></font><font color=blue><B>54</B></font><font color=black>f3</B></font> <font color=green><B>a4</B></font><font color=red><b>15</B></font><font color=blue><B>00</B></font><font color=black>f3</B></font> <font color=blue><B>01</B></font><font color=black>06</B></font><font color=blue><B>02</B></font><font color=black>04</B></font> <font color=blue><B>1a</B></font><font color=black>01</B></font><font color=blue><B>1b</B></font><font color=black>05</B></font> <font color=blue><B>1c</B></font><font color=black>04</B></font><font color=blue><B>53</B></font><font color=black>f3</B></font> <font color=red>4580ns</font>
<font color=blue><B>54</B></font><font color=black>05</B></font><font color=blue><B>55</B></font><font color=black>f3</B></font> <font color=blue><B>60</B></font><font color=black>01</B></font><font color=blue><B>61</B></font><font color=black>06</B></font> <font color=blue><B>62</B></font><font color=black>03</B></font><font color=blue><B>68</B></font><font color=black>f3</B></font> <font color=blue><B>69</B></font><font color=black>06</B></font><font color=blue><B>6a</B></font><font color=black>f3</B></font> <font color=blue><B>73</B></font><font color=black>f3</B></font><font color=blue><B>74</B></font><font color=black>07</B></font> <font color=blue><B>75</B></font><font color=black>03</B></font><font color=green><B>a3</B></font><font color=red><b>15</B></font> <font color=blue><B>00</B></font><font color=black>f3</B></font><font color=blue><B>01</B></font><font color=black>07</B></font> <font color=blue><B>02</B></font><font color=black>f3</B></font><font color=blue><B>0c</B></font><font color=black>03</B></font> <font color=red>5500ns</font>
<font color=blue><B>0d</B></font><font color=black>06</B></font><font color=blue><B>0e</B></font><font color=black>f3</B></font> <font color=blue><B>27</B></font><font color=black>04</B></font><font color=blue><B>28</B></font><font color=black>05</B></font> <font color=blue><B>29</B></font><font color=black>f3</B></font><font color=blue><B>45</B></font><font color=black>f3</B></font> <font color=blue><B>46</B></font><font color=black>06</B></font><font color=blue><B>47</B></font><font color=black>f3</B></font> <font color=blue><B>6e</B></font><font color=black>02</B></font><font color=blue><B>6f</B></font><font color=black>06</B></font> <font color=blue><B>70</B></font><font color=black>01</B></font><font color=blue><B>7c</B></font><font color=black>f3</B></font> <font color=blue><B>7d</B></font><font color=black>06</B></font><font color=blue><B>7e</B></font><font color=black>f3</B></font> <font color=green><B>a2</B></font><font color=red><b>15</B></font><font color=blue><B>29</B></font><font color=black>03</B></font> <font color=red>6260ns</font>
<font color=blue><B>2a</B></font><font color=black>13</B></font><font color=blue><B>2b</B></font><font color=black>0f</B></font> <font color=blue><B>2c</B></font><font color=black>01</B></font><font color=blue><B>2d</B></font><font color=black>06</B></font> <font color=blue><B>2e</B></font><font color=black>f3</B></font><font color=blue><B>3e</B></font><font color=black>f3</B></font> <font color=blue><B>3f</B></font><font color=black>06</B></font><font color=blue><B>40</B></font><font color=black>f3</B></font> <font color=blue><B>41</B></font><font color=black>f3</B></font><font color=blue><B>42</B></font><font color=black>06</B></font> <font color=blue><B>43</B></font><font color=black>04</B></font><font color=blue><B>48</B></font><font color=black>f3</B></font> <font color=blue><B>49</B></font><font color=black>08</B></font><font color=blue><B>4a</B></font><font color=black>f3</B></font> <font color=blue><B>4c</B></font><font color=black>f3</B></font><font color=blue><B>4d</B></font><font color=black>06</B></font> 
<font color=blue><B>4e</B></font><font color=black>05</B></font><font color=blue><B>4f</B></font><font color=black>05</B></font> <font color=blue><B>50</B></font><font color=black>f3</B></font><font color=blue><B>58</B></font><font color=black>f3</B></font> <font color=blue><B>59</B></font><font color=black>06</B></font><font color=blue><B>5a</B></font><font color=black>01</B></font> <font color=blue><B>67</B></font><font color=black>04</B></font><font color=blue><B>68</B></font><font color=black>05</B></font> <font color=blue><B>69</B></font><font color=black>f3</B></font><font color=red><b>a1</B></font><font color=red><b>15</B></font> <font color=red><b>04</B></font><font color=black>02</B></font><font color=blue><B>05</B></font><font color=black>05</B></font> <font color=blue><B>06</B></font><font color=black>01</B></font><font color=blue><B>36</B></font><font color=black>03</B></font> <font color=blue><B>37</B></font><font color=black>05</B></font><font color=blue><B>38</B></font><font color=black>f3</B></font> <font color=red>7320ns</font>
<font color=blue><B>3d</B></font><font color=black>01</B></font><font color=blue><B>3e</B></font><font color=black>05</B></font> <font color=blue><B>3f</B></font><font color=black>f3</B></font><font color=blue><B>41</B></font><font color=black>01</B></font> <font color=blue><B>42</B></font><font color=black>07</B></font><font color=blue><B>43</B></font><font color=black>01</B></font> <font color=blue><B>46</B></font><font color=black>f3</B></font><font color=blue><B>47</B></font><font color=black>05</B></font> <font color=blue><B>48</B></font><font color=black>01</B></font><font color=blue><B>4a</B></font><font color=black>f3</B></font> <font color=blue><B>4b</B></font><font color=black>05</B></font><font color=blue><B>4c</B></font><font color=black>f3</B></font> <font color=blue><B>4d</B></font><font color=black>02</B></font><font color=blue><B>4e</B></font><font color=black>07</B></font> <font color=blue><B>4f</B></font><font color=black>03</B></font><font color=blue><B>5d</B></font><font color=black>f3</B></font> 
<font color=blue><B>5e</B></font><font color=black>06</B></font><font color=blue><B>5f</B></font><font color=black>06</B></font> <font color=blue><B>60</B></font><font color=black>01</B></font><font color=blue><B>6c</B></font><font color=black>02</B></font> <font color=blue><B>6d</B></font><font color=black>06</B></font><font color=blue><B>6e</B></font><font color=black>f3</B></font> <font color=blue><B>7c</B></font><font color=black>02</B></font><font color=blue><B>7d</B></font><font color=black>05</B></font> <font color=blue><B>7e</B></font><font color=black>f3</B></font><font color=red><b>a0</B></font><font color=red><b>15</B></font> <font color=red><b>07</B></font><font color=black>f3</B></font><font color=blue><B>08</B></font><font color=black>05</B></font> <font color=blue><B>09</B></font><font color=black>f3</B></font><font color=blue><B>0c</B></font><font color=black>f3</B></font> <font color=blue><B>0d</B></font><font color=black>06</B></font><font color=blue><B>0e</B></font><font color=black>f3</B></font> <font color=red>8600ns</font>
<font color=blue><B>0f</B></font><font color=black>05</B></font><font color=blue><B>10</B></font><font color=black>03</B></font> <font color=blue><B>27</B></font><font color=black>01</B></font><font color=blue><B>28</B></font><font color=black>06</B></font> <font color=blue><B>29</B></font><font color=black>f3</B></font><font color=blue><B>3c</B></font><font color=black>01</B></font> <font color=blue><B>3d</B></font><font color=black>06</B></font><font color=blue><B>3e</B></font><font color=black>01</B></font> <font color=blue><B>57</B></font><font color=black>01</B></font><font color=blue><B>58</B></font><font color=black>06</B></font> <font color=blue><B>59</B></font><font color=black>f3</B></font><font color=blue><B>70</B></font><font color=black>01</B></font> <font color=blue><B>71</B></font><font color=black>05</B></font><font color=blue><B>72</B></font><font color=black>f3</B></font> <font color=blue><B>7f</B></font><font color=black>03</B></font><font color=yellow><B>c1</B></font><font color=yellow><B>c1</B></font> 



<!-- Comment: Fri Jan 18 09;01;24 comment by...Lester -->
<pre>Typo: I meant SB0W4L3 not SB0W3L4 (OBDV)
</pre>


<!-- Comment: Wed Jul 21 10;21;41 comment by...    -->
</pre>

<!-- Date: Fri Jan 18 09:07:38 2002 -->
<pre>the trigger table we're now using is much different than the one 
  used for 128501, from which i was deriving the number of needed 
  events.  aaron, jason took a quick look at a small part of run 
  137340 and i spent some time trying to understand the old and new 
  physics trigger tables.  the upshot is that we think we only need 
  1/2 as many events as previously listed. 
 
  the special run list now looks like this: 
 
 
              SVX/ISL     SVX 
        L00   PHI/SAS   90-DEG    DPS    NEVENTS       STATUS   1st RUNNO 
        ===   =======   ======    ===    =======      ========  ========= 
     ----- take dps-on and dps-off calibration ------ 
     1.  3       5         4      OFF     225k         DONE      137340 
     2.  3       5         4       ON      60k                   137387 
 
     ----- take dps-on and dps-off calibration ------ 
     3.  4       6         5      OFF     225k 
 
     ----- take dps-on and dps-off calibration ------ 
     4.  2       7         6      OFF     225k 
 
     ----- take dps-on and dps-off calibration ------ 
     5.  5       4         3      OFF     225k 
 
     ----- take dps-on and dps-off calibration ------ 
     6.  3       5         4       ON     225k  (if time) 
 
          &gt;&gt;&gt; care should be taken that CALIBRATIONS be made 
          &gt;&gt;&gt; BEFORE EACH of the different BW RUNS !!! 
 
                - need to take dps-on AND dps-off calibration for 
                  each BW set; after talking w/ Igor, we decided 
                  to use 
 
                        600 evts for dps-off calib 
                       1200 evts for dps-on calib 
                  since we'll have to use the software EVB, this 
                  smaller number of events could speed things up 
                  a bit 
 
          &gt;&gt;&gt; the BW is changed ONLY on these special wedges/layers 
          &gt;&gt;&gt; the rest of the detector is w/ "normal" settings 
 
        - set the following wedges/layers in RA 
 
                S-B3W8L0:4  (SVT has masked this wedge off) 
                S-B5W8L0:4  (SVT has masked this wedge off) 
                I-B2W0L4 
                I-B1W4L4 
                I-B1W5L0,L2 
                any available L00 
 
        - during collisions 
 
        - detector bias is necessary 
 
        - parasitic to normal data taking 
 
        - require track-trigger, magnet ON, "dflt" physics trigger table 
 
  - if the silicon gets "handshaked" (handshook?) in the middle of the run, 
    YOU NEED TO PERFORM A COLDSTART in order to recover the special ISL 
    low power settings.  THIS IS IMPORTANT!!! 
 
 - when you're finished, set all L00 to BW=3, DPS=OFF, RA 
                          set all ISL to BW=5, DPS=ON,  NN 
                          set all SVX to dflt (i think, bw=5, dps=on, nn) 
 
 
 
</pre>
<!-- Author: Jean -->
<!-- Comment: Fri Jan 18 16;29;10 comment by...David -->
<pre>The handshake warning is true for all chips, not just the special ISL low power settings. A handshake will mess up all chips (polarity for example).
</pre>
<!-- Date: Fri Jan 18 15:38:49 2002 -->
<pre>Calibrations taken: 
	- dps-on  = 137447 
	- dps-off = 137448 
 
 
RA HDIs set as BW(L00/phi-SAS/90-deg) = (2/7/6), which is #4 
in the run plan given above.   
 
The rest of the detector has dflt settings. 
 
</pre>
<!-- Author: Doug -->


<!-- Date: Fri Jan 18 15:57:31 2002 -->
<pre>we're about 1/2-way through.  unstable beam kept us from powering for 
  the first 7 or so hours of last night's run.  too bad.  next shot at 
  1200ish today.  we'll stay at the pt. 3 settings for the 
  remainder of the present store and change to the pt 4 settings 
  for the next shot.  hopefully we can polish the list off with this next 
  store. 
 
  here are the relevant run numbers (so far): 
 
        ------ BW SETTING ----- 
              SVX/ISL     SVX 
        L00   PHI/SAS   90-DEG    DPS    NEVENTS      STATUS  1st RUNNO 
        ===   =======   ======    ===    =======      ======  ========= 
     1.  3       5         4       ON    CALIB         DONE     137337 
     1.  3       5         4      OFF    CALIB         DONE     137338 
     1.  3       5         4      OFF     225k         DONE     137340:46 
     2.  3       5         4       ON      60k         DONE     137416 
 
     3.  4       6         5       ON    CALIB         DONE     137417 
     3.  4       6         5      OFF    CALIB         DONE     137418 
     3.  4       6         5      OFF     225k      in progress 137419, 
 
     4.  2       7         6       ON    CALIB 
     4.  2       7         6      OFF    CALIB 
     4.  2       7         6      OFF     225k 
 
Igor, can you please ensure the calibrations are appropriately 
   processed?  once they are, can you please email aaron and jason so that 
   they can start processing a sub-set of the first two runs?  we need to 
   first validate the new ntuple (if jason hasn't already done so) and 
   verify that the dps-emulation works.  i think if we look at S-B3W8 L2 
   and L3 for ~30k STREAM-B events, we ought have sufficient stats to 
   make these decisions (will yield ~500 golden clusters on each side of 
   L2 and L3). 
</pre>
<!-- Author: Doug&#39;s email -->


<!-- Date: Fri Jan 18 16:06:03 2002 -->
<pre>SB0W6L2 (AVDD2 PROBLEM) did not initialized. Init masked by Lester.</pre>
<!-- Author: Saverio -->


<!-- Date: Fri Jan 18 16:07:49 2002 -->
<pre> 
SB4W7 tripped for external trip at 14:19</pre>
<!-- Author: Saverio -->



<!-- Date: Fri Jan 18 22:04:37 2002 -->
<pre>From The main log: 
 
    Fri Jan 18 21:39:23  
 
Taking DPS-on silicon D-mode calibration with the following BW  
settings:  
 
L00: 5  
SVX/ISL phi, sas: 4  
SVX 90-deg: 3  
 
 
 - Andy H. :: (run 137471)  
 
    Fri Jan 18 21:41:50  
 
Same thing, but DPS off.  
 
 - Andy H. :: (run 137472)  
</pre>
<!-- Author: L -->


<!-- Date: Fri Jan 18 23:43:15 2002 -->
<pre>I just did an unmarking sweep of Imon (one at a time -- I 
remember at one point "Unmark all" didn't work -- still true?) 
 
The following had persistant pinkies ( * means it can be ignored 
according to the post-it notes on b0dap42): 
 
SB0W2 L0 AVDD (*) 
SB0W2 L1 AVDD (*) 
SB0W2 L3 DVDD (*) 
SB0W6 L2 AVDD 
SB5W3 L4 DVDD 
SB5W8 L2 DVDD 
SB5W8 L4 DVDD 
SB1W4 L4 DVDD (doesn't return as quickly as the others) 
 
LB0W5 L3 DVDD 
LB1W4 L1 DVDD 
LB1W4 L3 DVDD 
 
IB1W5 L0 DVDD 
IB1W5 L2 DVDD 
IB2W0 L4 DVDD 
 
Note that only the SB0W2's are listed as "known," and 
a bunch of others that ARE listed appear to be OK now.  If we 
really want aces to take Imon-watching seriously, the post-its 
ought to reflect reality a little more...</pre>
<!-- Author: Andy H. -->
<!-- Comment: Sat Jan 19 04;58;25 comment by...aces (misc.) -->
<pre>See the main e-log for more (than you want to know) about these
channels.</pre>
<!-- Date: Sat Jan 19 02:06:19 2002 -->
<pre>cratematch excerpt below: 
 
The following wedges should be biased and are not:  
 
SB2W5L0  SB2W5L1  SB2W5L2  SB2W5L3 
 
CAENvolt.dat excerpt below indicates these ladders should be 
biased: 
 
B2W5                                                                            115.0 1 A                                                                       90.0 2 A                                                                        49.0 3 B                                                                        90.0 4 A                                                                        40.0 5 B x 
 
There is nothing in the e-log I can find to indicate that these 
ladders should not be biased 
 
Power was turned on as below (from GUI log) 
Fri Jan 18 17:29:14 CST 2002 success: Power ON  
There are no subsequent messages indicating a failure  
to turn on SB2W5, but this appears to be what happened. 
 
The GUI needs to be fixed so this doesn't happen.  The relevant 
people have been informed of this type of problem and are working 
towards a solution. 
Until then, these ladders will be left unbiased. </pre>
<!-- Author: Chris Hill -->


<!-- Date: Sat Jan 19 04:04:10 2002 -->
<pre>Changed back to L00/PHI/90 3/5/4 DPS on 
Aces are taking calibrations. 
</pre>
<!-- Author: Lester -->



<!-- Date: Sat Jan 19 11:45:29 2002 -->
<pre>Changed silicon to nominal configuration 
 
L00 DPS off RA BW3 
ISL DPS on  NN BW5 
SVX DPS on  NN BW5 
 
first run with new configuration is 137486 
Aces are instructed to take calibrations during quiet time. 
Shot dump at noon. 
</pre>
<!-- Author: Lester -->
<!-- Comment: Sat Jan 19 15;38;43 comment by...Doug -->
<pre>Excellent!  This completes the RA/BW scan runs.  All the SVX/
ISL RA wedges should be dps-on NN.  Lester, if this is true,
can you please page the SVT and tell them that they can unmask
s-b3w8 and s-b5w8?

I'll look through the run summaries and post a message with the
relevant run numbers for each scan point.</pre>


<!-- Date: Sat Jan 19 16:27:59 2002 -->
<pre> 
Here are the relevant runs for the BW/RA scans.  I'm only 
listing runs w/ &gt;1/nb of data.  These are the runs Aaron/Jason 
ought process once the calibrations are in place. 
 
   ---BW Setting --- 
   L00  phi/sas   90    dps   lumi    run nos. 
 ====================  ===== ======   ===================== 
1.  3      5       4     ON   calib    137337 
    3      5       4    OFF   calib    137338 
    3      5       4    OFF   34/nb    137340:346 
2.  3      5       4     ON   14/nb    137416 
 
3.  4      6       5     ON   calib    137417 
    4      6       5    OFF   calib    137418 
    4      6       5    OFF   19/nb    137419 (may need more) 
 
4.  2      7       6     ON   calib    137447 
    2      7       6    OFF   calib    137448 
    2      7       6    OFF   36/nb    137465:468, 470 
 
5.  5      4       3     ON   calib    137471 
    5      4       3    OFF   calib    137472 
    5      4       3    OFF   37/nb    137475:478 
 
6.  3      5       4     ON   calib    137479 
    3      5       4    OFF   calib    137482 
    3      5       4     ON   36/nb    137484:485 
 
Once we've taken a look at some of these, we'll know whether or 
not we ought take additional data at pt 3. 
</pre>
<!-- Author: Doug -->


<!-- Date: Sat Jan 19 17:40:42 2002 -->
At 17:12 there was a PS GUI alarm for L00 B1W5 L4 ("reported 
OFF but low/high voltage is above 2/5 V").  It went away and 
a check of this wedge showed nothing unusual.  <b>But last time 
I checked, L00 only has 3 "layers" -- what's up with that?</b>  
<!-- Author: Andy H. -->
<!-- Comment: Sat Jan 19 17;42;12 comment by...Andy H. -->
<pre>Well, OK, actually 4 "layers," but numbered 0-3, so what's "L4?"</pre>


<!-- Date: Sat Jan 19 17:45:02 2002 -->
<pre>At 17:16 there was a software protection trip for SB5W2 L0 DVDD. 
Paged Dr. Hill and turned it back on.</pre>
<!-- Author: Andy H. -->
<!-- Comment: Sat Jan 19 18;01;52 comment by...CH -->
<pre>This was the second DVDD software protection trip today on SB5W2.
DVDD current limits may need to be increased. 

Will the next diagnostic shift please look into this wedges
current history.  

GUI log excerpts below:

Sat Jan 19 00:45:24 CST 2002 Software protection trip: High DVDD Current SVX B5 W2 L2
Sat Jan 19 00:45:37 CST 2002 Software protection trip: High DVDD Current SVX B5 W2 L2

Sat Jan 19 17:16:52 CST 2002 Software protection trip: High DVDD Current SVX B5 W2 L2
Sat Jan 19 17:17:06 CST 2002 TRIP: SVX B5 W2 DVDD0 OverCurrent 
</pre>


<!-- Comment: Sat Jan 19 23;19;17 comment by...Andy H. -->
<pre>And again at 22:46... see below</pre>
<!-- Date: Sat Jan 19 18:57:41 2002 -->
<pre>Summary of Initialization failures for AVdd-2 probem ladders 
 
 
SB3W3L4 
 first data  01-16-2002 23:09:03  Fib e334 hdi 9 ERRORS 
 init masked 01-16-2002 23:39:40  Fib e334 hdi 19 ERRORS 
 last data   01-19-2002 17:30:08  Fib e334 hdi 19 ERRORS 
 I found 0 successes and 302 failures since 01-16-2002 
 
SB4W3L2 
 first data 01-16-2002 23:09:05 Fib e432 hdi 17 ERRORS 
 last data  01-19-2002 17:30:10 Fib e432 hdi 17 ERRORS 
 I found 0 successes and 300 failures since 01-16-2002 
 
SB1W8L2 
  first data    12-14-2001 23:23:17 Fib e182 hdi 12 ERRORS 
  first Success 12-15-2001 16:24:44 Fib e182 hdi 12 SUCCESS 
  until         12-19-2001 12:20:32 Fib e182 hdi 12 ERRORS 
  until         01-16-2002 23:31:21 Fib e182 hdi 12 SUCCESS  
  last data     01-19-2002 17:30:11 Fib e182 hdi 12 SUCCESS  
  I found 472 successes and 29 failures since 12-14-2001  
 
SB0W6L2 
  first data    01-16-2002 23:09:04 Fib e062 hdi 2 ERRORS 
  init masked   01-18-2002 15:56:47 Fib e062 hdi 12 ERRORS  
  last data     01-19-2002 18:33:26 Fib e062 hdi 12 ERRORS 
  I found 0 successes and 100  failures since 01-16-2002 
 
SB0W0L2 
  first data    12-14-2001 18:21:19 Fib e002 hdi 12 ERRORS 
  succes on     12-15-2001 19:54:41 Fib e002 hdi 12 SUCCESS 
  and on        12-15-2001 19:54:46 Fib e002 hdi 12 SUCCESS 
  last data     01-19-2002 18:33:26 Fib e002 hdi 12 ERRORS 
  I found 2 successes and 970 failures since 12-14-2001 
 
SB2W9L4  
  first data    12-20-2001 15:04:56  Fib e294 hdi 9 ERRORS 
  last data     01-01-2002 01:44:02  Fib e294 hdi 9 ERRORS 
  I found 0 successes and 9 failures since 12-20-2001 
</pre>
<!-- Author: Saverio -->


<!-- Date: Sat Jan 19 23:18:37 2002 -->
<pre>Yet another SB5W2 L0 DVDD overcurrent software protection trip 
at 22:46.  In lieu of harrassing Chris again, I just turned it 
back to on and then the DAQ ace cold-started b0fib04. 
</pre>
<!-- Author: Andy H. -->



<!-- Date: Sun Jan 20 03:43:09 2002 -->
<pre>Store was just lost and SB0W5 left ON (biased). 
   
This happened because shift crew thought they had turned  
everything off with the global off switch (in this case they 
used IFIX HV control) and SB0W5 didn't respond to the command. 
 
It has happened before that some ladders have been turned on  
manually after a global on failed to succeed and that these 
same ladders need to be shut off manually as well. 
 
This may have been what happened to SB0W5, although no evidence 
of this from the PS log. 
 
Shift crews should be instructed to turn HV on globally only, 
preferably from IFIX, and whatever ladders do not come up biased  
should be left in standby until this problem is understood. 
 
We need to add some audible warnings to the system when global 
standby doesn't succeed.  
 </pre>
<!-- Author: Chris Hill -->


<!-- Date: Sun Jan 20 10:03:36 2002 -->
<pre>SB0W6L2 AVDD is persistently pink. 
nominal value= 770.0 
current = 690.0. 
probably minor, but to your attention.</pre>
<!-- Author: aces on duty (Muge/Yamashita) -->


<!-- Date: Sun Jan 20 11:23:04 2002 -->
<h3>More X Mode results</h3> 
I wrote the results of 137372/137373 X mode calibration 
runs taken <a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=92&frame=2&anchor=13:35:41'> 
here</a> into the online production database.  It is 
<pre> 
Wrote ValidSet 53380 to DB onotl_prd for process FIB_PHYSICS_SVX 
SICHIPPED run number    = 137373 
SIDPSSSHIFT run number  = 137373 
SICHIPTHRESH run number = 137373 
SISTRIPDH run number    = 9999999 
SIFIBOFFSETS run number = 9999999 
</pre>  
This is under process name FIB_PHYSICS_SVX-In principle, this 
should have been done Thursday, and in the future in a more 
automated fashion, just after the calibration is taken. 
On the other hand, this will supercede an entry from Dec 30, so 
I figure it cannot be any worse than something that old. 
HDI Error logs for those X Mode runs and an explanation 
are <a href= 
'/cgi/elog/det-system-elog.pl?nb=barrel4&action=view&page=51&frame=2&anchor=11:19:28'>here</a> in the 
Barrel 4 elog. We are clearly running some ladders which don't work for this or that reason.  
<!-- Author: SCN -->


<!-- Date: Sun Jan 20 17:30:17 2002 -->
<pre>I tested biasing SB0W7L4 a standby ladder, got the following: 
(first reading at that voltage) 
V	I 
------------ 
2	35 
5	124 
10	241 
20	429 
30	547 
directly to 30 from 2 draws 552, drops from there. 
 
I leave it biased and changed CAENvolts.dat by hand (NOT HWDB!) 
Tomorrow we can look at the currents and see if this should be  
permament. 
</pre>
<!-- Author: Lester -->


<!-- Date: Sun Jan 20 20:21:23 2002 -->
<pre>ACE having trouble with SVXMon. 
Consumer expert says problem is with SVXMon and advises SVXMon  
not be run. 
 
Thus, we are running without SVXMon until SVXMon experts can fix 
the problem (we will have to rely on our other monitors for  
things like bias problems). 
 
I know of know pager for SVXMon experts, but have e-mailed Igor. </pre>
<!-- Author: Chris Hill -->
<!-- Comment: Mon Jan 21 01;36;16 comment by...CH -->
<pre>Igor has fixed the problem and SVXMon is back online.</pre>
<!-- Date: Mon Jan 21 01:31:24 2002 -->
<pre>Software trip (DVDD overfcurrent) in SB5W2L2. Mircea reset the trip with Chris's directions. SB5 is now all green.</pre>
<!-- Author: Slava -->
<!-- Comment: Mon Jan 21 01;42;29 comment by...CH -->
<pre>This ladder continues to trip off.  It needs to have its current
limits adjusted.  It also needs to be understood why it goes
into the high current states.</pre>

<!-- Comment: Mon Jan 21 02;56;27 comment by...Slava -->
<pre>SB5W2 tripped again. Under investigation.</pre>


<!-- Date: Mon Jan 21 03:21:16 2002 -->
<pre>Resetting trip on DVDD highC in SB2W10L4 
Also, noticed "ALARM: LB1W5L4 reported off but HV is &gt; 2/5V" in PS GUI</pre>
<!-- Author: Slava -->


<!-- Date: Mon Jan 21 06:42:55 2002 -->
<pre>Silicon: PS10 appeared to be left off after SB2W10L4 DVDD highCurrent trip recovery. Putting on standby... 
Got DVDD highC trips in SB2W10L3, try again with PS10--&gt;STANDBY</pre>
<!-- Author: Slava/Mircea -->


<!-- Date: Mon Jan 21 11:10:45 2002 -->
<pre>Run Plan 
-------- 
1. quiet time while hunting for ground fault 
2. Ilya - hardware event builder and x-mode debugging 
3. Steve Nahn - code cleanup 
4. missing BW scan calibration - Lester 
5. Change ISEL sbits on AVDD2 candidates 
6. once beam is back, long runs for alignment 
7. turn off avdd2 ladders 
</pre>
<!-- Author: Jean, Steve,  Lester -->
<!-- Comment: Mon Jan 21 15;48;55 comment by...Doug -->
<pre>i thought all the BW runs had the appropriate calibration.  
perhaps i mis-read the elogs.  at any rate, lester, can you
please appropriately comment the entry above so that we have a
record of which runs go with which scan point?  thanks.
</pre>
<!-- Date: Mon Jan 21 12:59:35 2002 -->
<h3>Squashed a 4 year old bug!</h3> 
Turns out in the chip initialization routines written 
by Wolfram and Jean in 1998, the last bit read back from 
the fib was always set low (becuase they used 0x7f, not 0xff, 
as a mask).  This means that the last chip would fail if 
and only if you set rampped > 8.  This bug survived all  
SiDet testing, all integration, but has now been squashed into 
a little ball of slimy bug goo.  David is adjusting L00 
ramppeds already.
<!-- Author: SCN -->


<!-- Date: Mon Jan 21 14:51:32 2002 -->
<h3> L3 happier with D Mode </h3> 
Seems new L3 code installed by Ilya today is happier 
with Silicon calibrations (<a href= 
'/cgi/elog/elog.pl?nb=2002&action=view&page=62&frame=2&anchor=14:07:48'>Details</a>).  Took 
D Mode calibrations at 22 MB/sec <ul> 
<li> DPS OFF 137627 
<li> DPS ON  137630 
</ul> 
Everything (including L00) was biased and L00 has new 
rampped settings.  In the process, got VBIAS External Trips 
on SB0W4 L0 and SB5W0 L2.  Turning them back on worked.  
  
<!-- Author: LHM, SCN -->


<!-- Date: Mon Jan 21 15:37:55 2002 -->
<pre>Preliminary study of Avdd current as a function of time after early December has been done. The following ladders have decreasing current: 
SB0W0 (except for L2), SB0W1, SB0W4L4, SB0W5L3, SB0W6L1, SB0W6L4, 
SB0W7, SB1W1, SB1W3L3, SB1W6 (except for L1), SB1W8L4, SB1W9,  
SB1W10L3, SB1W11L4, SB2W1 (except for L4), SB2W2L1, SB2W2L3,  
SB2W7L2, SB2W11,SB3W1, SB3W5 (except for L0 and L1), SB3W9L3,  
SB3W9L4, SB4W2L3, SB4W6 (except for L2 and L3), SB4W10L2, and  
SB5W9L3. 
 
The followings have increasing current: 
SB0W3, SB1W2L2, SB1W2L4, SB4W4L2, and SB5W3L1 
 
Others are inconclusive due to lack of data. 
 
There seemed a change of current before Dec.25 and after Jan 5. 
But the amount of data is not conclusive.</pre>
<!-- Author: John Zhou -->



<!-- Date: Mon Jan 21 18:06:37 2002 -->
<pre>Looking at this weekends data, I see several ISL chips that need to have BiasRatioSelect reverted to 1. They are now back. 
Affected chips are: f014_15, f034_14-15, f114_15, f120_8-9, f400_5,9, f430_4, f452_4, f500_11,9, f501_14 
 
where the nomenclature is HDI_chip and chips are numbered in readout order. 
 
</pre>
<!-- Author: David -->


<!-- Date: Mon Jan 21 19:19:07 2002 -->
<b>Silicon Power Supply Stuff Done on Today's Controlled Access:</b> 
<ul> 
<li>Checked the bias cables for IB0W4.  They are fine.  Perhaps the mistake is at the JC? 
<li>Verified that LB1W5 is indeed putting out 5.11V on its 5V DOIM at all times(i.e. it is malfunctioning).  Since this is our rad-hard prototype I decided to leave it in for the time being.  It is not cabled up, so it is not hurting anything. 
</ul>
<!-- Author: Napora -->
<!-- Comment: Tue Jan 22 11;28;44 comment by...herndon -->
<pre>This looks like another confirmed failure of the Micrel.  Will
notify CAEN and other interested parties.  Probably we would like
to discuss interlock proposals again at this point.</pre>
<!-- Date: Mon Jan 21 22:57:42 2002 -->
<h3>Access Results</h3> 
Lester and I tried to reproduce some problems on 
troubled wedges, then swap the Timing cables and 
see if we can trace the problems to one or the other 
side of the Timing Cable.  Our Results: 
<ul> 
<li> SB0W2/SB0W3: Wedge  <a href= 
'/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=91&frame=2&anchor=19:04:06'>oscillates</a> between good and bad state.  Was not able to reproduce bad state 
with svxdaq, but swapped out fib anyway.  Out: FIB 126 560611 
In but didn't work: FIB 112 560608 (Readout screwed up?) In Finally FIB 110 560610 put in, same ladders still work.  Will 
need to watch to see if Fib swap fixes it. 
<li> SB2W4/SB2W5: SB2W4 doesn't initialize, currents are bad.  Swapped Timing Cables, and SB2W5 worked 
in both spots.  SB2W4 worked "better" in that it read 
back most of the initialization correct, so we swapped it back, 
and found that wiggling the command cable could make it read 
back better or worse.  Left it in a state where it doesn't 
read back the init correctly, but does read out ok (at least at standby) so could be used.  Suspect we should replace the 
Fib Timing Cable, or reterminate the FTM connector. 
<li> SB5W2/SB5W3: Reproduced the problem of all CAP Ids out of synch in entire wedge, but only occasionally (i.e. not every run).  SRC HRR did not fix it when it occurred.  Swapped Timing 
Cables and problem was not seen.  Swapped back, and problem 
was again not seen.  Flaky connection? 
<li> SB1W4/SB1W5 - Could not reproduce the Vrb header problems 
with svxdaq.  Do not believe this is a FIB problem anyway. 
</ul> 
Along the way also the Aces rebooted our fib crate in the  
middle of a firmware update, and b0dau30 died...sigh.  But 
maybe we re-rescued SB2W4 (at least for awhile).  
<!-- Author: SCN -->
<!-- Comment: Mon Jan 21 23;25;19 comment by...Lester -->
<pre>I reintegrated the wedge SB2W4 so we can track the progress of
the 'fix'. It has all init readback masked off for now.
</pre>


<!-- Comment: Tue Jan 22 11;26;40 comment by...herndon -->
<pre>Reterminating the FTM end is almost impossible.  Replacing the
cable is no problem with access.</pre>
<!-- Date: Tue Jan 22 09:20:56 2002 -->
<table border=0><td><tr><td bgcolor=#e0e030>a few complaints from your friendly ACEs. (Jean asked me to post this as a follow up to  
<a href= 
'/cgi/elog/elog.pl?nb=2002&action=view&page=65&frame=2&anchor=08:35:07'> 
this entry</a> from main elog). 
First, the instruction for the ACEs that we can find from the ACE help webpage are outdated. Specifically, I noticed the following: <br>1. info on restarting the silicon GUI is incomplete  
<br>2. the allowed losses level is not updated (is the  limit of 25kHz still current?) 
<br>3. No instructions are present on what to do with the IMON. At different points in time, ACEs were asked to do different things about IMON, pink channels, etc. Information about IMON and    
things you want ACEs to do in there, should be posted on the help page. 
<br>4. Instructions on <font color=red>how</font> to do D-mode calibrations should be linked to from the ACEs' calibration page - it's the place where _all_ the calibration procedures can be normally found. <br>Instructions on <font color=red>when</font> to do D-mode calibrations are not present at all. 
<br> 
<br>There may be other obsolete things as well. In any case, the old shift crews are finishing their round of shifts, so you have a good reason to update the webpage - that's where new ACEs will turn for information when something goes wrong.  
<br>Second, the things we ACEs were asked to do about silicon sometimes changed. An example would be the allowed radiation loss level (during my time alone, I was told to watch for 30kHz, 20kHz and then 25kHz of LOSTP).  Another example is those infamous (among ACEs) pink channels in IMON - ACEs have been asked to do all kinds of things with them - from unamrking and noting in elog every single one, to ignoring them completely. My comment is that it would be good to have a place where the ACEs could look for information about the latest policy on silicon. The page could be very short and include only 2 or 3 items but it would be extremely useful. 
<br> 
Anyway, I hope you don't take this entry as a lot of whining, I am just trying to make life easier for you and for our new ACEs. If you get paged less, you experts get more sleep, and we ACEs resolve the problems faster. </td></tr></td></table>
<!-- Author: Denys (retiring ACE) -->




<!-- Date: Tue Jan 22 16:36:26 2002 -->
    <h1>PRELIMINARY WARNING</h1> 
    What follows has lots and lots of detail.  It tries to methodically search for 
    what in the silicon may still cause events to be dropped because of format, at least for 
    a particular run range.  The rate of rejection has been under 1% since the new Fib firmware, 
    which just means we swept the major problems away from the sensitivity of the 
    reformatter.  Still<ul> 
      <li> The errors which cause the RF to toss the event may be correlated with other 
	not very well understood errors where the events are kept (i.e. "00f3") 
      <li> If there is something we can do about these (i.e. Fib firmware, sequences, etc) we 
	should do it. 
    </ul> 
     
    <h1>Reformatter Errors as of late</h1> 
    I grepped my way through log files for Run 137000-current (137712) or 
    so, in which the Si took on the order of 70.7 million L1A overall, and 
    CDF took 69.9 million L1A and 10.3 million L2A in BEAM + PHYSICS 
    mode-don't know how much of it was with Silicon (don't have all the 
    right queries, but those are just ot give a feel for numbers). 
    Upon every L2A the data flows to the Reformatter, which has the 
    potential to throw away the event if it finds an error. I found <ul> 
      <li> 8881 Fib ID errors (RF keys them as <I>UTILSILDEC</I>) 
      <li> 2685 Byte Count errors (<I>DLINCO</I>) 
      <li> 17583 Fib ID Imposters (<I>BNK-INSTAN</I>) 
      <li> 23 Status Word Errors (<I>STAICO</I>) 
      <li> 55 TSL1A/TS Mismatches (<I>VRBWRD</I>) 
    </ul> 
    Detailed descriptions and debugging follow. For those who just want to get to 
the next entry, go <A HREF="#theEndofTheEntry">here</A> 
    <ul> 
      <li> <h4>UTILSILDEC: FIB ID errors- Total 8881</h4>This is when the RF looks at the first word in 
	an HDI block (cf. CDF4152) to figure out which HDI this is from, and 
	doesn't understand what it sees.  Note we only have records for what the odd wedges Fib Ids were from 
	after the reformatter fix, in Run 137340 or so.   
	They fall into several catagories<ol> 
	  <li> <I>Chip ID, not Fib ID</I> <B>Total: 7965</B><BR> This is when what the RF sees 
	    looks not like a Fib ID, but rather a Chip ID, in fact the first Chip ID 
	    for that HDI, as in this example: 
	    <pre>FIB ids found: 
  0x831f0004  0x851f0009  0x891f0006  0xadeadcdf  0x8d1f0003  0x831f00f3  0x851f0002  0x891f0007  0x891f0008  0x8d1f0004</pre> 
	    In this event, the individual channel byte counts add up to the 
	    total byte count for the Vrb, so it is probably really what the 
	    Fib sent.  Sometimes you do get a Fib header in at least one channel, 
	    so you can identify where the problem lies.  Note numbers do not sum to total, and  
	    many events have the problem with more than 1 channel in the Fib/Vrb. 
	    <table> 
	      <tr> <th>Wedge</th>  <th>L0</th> <th>L1</th> <th>L2</th> <th>L3</th> <th>L4</th> </tr> 
	      <tr> <td>SB0W2</td> <td> 0 </td> <td> 0 </td>  <td>135</td> <td> 0 </td> <td> 0 </td> </tr> 
	      <tr> <td>SB0W8</td> <td>880</td> <td>898</td> <td>862</td> <td>1773</td> <td>915</td>  </tr> 
	      <tr> <td>SB0W9</td> <td>348</td> <td>386</td> <td>345</td> <td>424</td> <td>386</td>  </tr> 
	      <tr> <td>SB3W7</td> <td>0 </td> <td>0</td> <td>0 </td> <td>0 </td> <td>1</td>  </tr> 
	      <tr> <td>LB0W4</td> <td>0 </td> <td>0</td> <td>0 </td> <td>1 </td> <td>N/A</td>  </tr> 
	    </table> 
	    By run, the rate seems to have dropped off, but I have not normalized to # L2A-If I kick 
	    out the small numbers I get: 
	    <table> 
	      <tr> <th> Run (Range) </th> <th> # Occurances </th> </tr> 
	      <tr> <td> 137023 </td> <td> 936 </td> </tr> 
	      <tr> <td> 137000-137299 except above </td> <td> 12 </td> </tr> 
	      <tr> <td> 137340</td> <td> 3142 </td> </tr> 
	      <tr> <td> 137341</td> <td>   36 </td> </tr> 
	      <tr> <td> 137343</td> <td> 2142 </td> </tr> 
	      <tr> <td> 137344</td> <td>  810 </td> </tr> 
	      <tr> <td> 137345</td> <td>  2 </td> </tr> 
	      <tr> <td> 137346</td> <td>  891 </td> </tr> 
	      <tr> <td> 137347</td> <td>  28 </td> </tr> 
	      <tr> <td> 137348</td> <td>  1 </td> </tr> 
	      <tr> <td> 137300-137399 except above </td> <td> 3 </td> </tr> 
	      <tr> <td> 137400-137499  </td> <td> 31 </td> </tr> 
	      <tr> <td> 137500-137559  </td> <td> 24 </td> </tr> 
	    </table> 
	    Someone pursuing this might look to see how many L2A were taken at what rate, and 
	    if any critical ladders were tossed out which brought the rate down. 
	  <li> <I> 0XC4C1 instead of Fib Header </I>: <B>Total: 528 </B><BR> 
	    This is exactly what it says: 0xc1c4 instead of a Fib header.  It only happens 
	    to a handful of ladders-breakdown is as follows: 
	    <table> 
	      <tr> <th> Ladder(s) </th> <th> # Occurances </th> </tr> 
	      <tr> <td> SB0W1 L2 </td> <td> 4 </td> </tr> 
	      <tr> <td> SB0W7 L2 </td> <td> 3 </td> </tr> 
	      <tr> <td> SB0W9 L1 </td> <td> 1 </td> </tr> 
	      <tr> <td> SB2W1 L3 </td> <td> 2 </td> </tr> 
	      <tr> <td> SB2W7 L0 </td> <td> 5 </td> </tr> 
	      <tr> <td> SB2W7 L4 </td> <td> 27 </td> </tr> 
	      <tr> <td> SB4W3 L2 </td> <td> 1 </td> </tr> 
	      <tr> <td> SB4W7 L2 </td> <td> 257 </td> </tr> 
	      <tr> <td> SB4W7 L4 </td> <td> 8 </td> </tr> 
	      <tr> <td> SB4W7 L2 & L4 </td> <td> 2 </td> </tr> 
	      <tr> <td> SB4W9 L2 </td> <td> 42 </td> </tr> 
	      <tr> <td> SB5W5 L2</td> <td> 54 </td> </tr> 
	      <tr> <td> SB5W5 L4 </td> <td> 2 </td> </tr> 
	      <tr> <td> SB5W5 L2 & L4 </td> <td> 2 </td> </tr> 
	      <tr> <td> SB5W7 L0 </td> <td> 117 </td> </tr> 
	      <tr> <td> IB1W5 L2 </td> <td> 1 </td> </tr> 
	    </table> 
	    The fib isn't suppossed to do this...we'll (Ken and I)  have to think about it. 
	  <li>Grab Bag: <I>Other "Fib ID" problems</I> <B>Total: 388</B><ul> 
	      <li> SB0W2 L2: e022 id becomes crap. <B>Total: 237</B><BR> 
		An example<pre>FIB ids found: 
  0xe0201bfe  0xe0211bfe  0xf9adf8ad  0xe0231bfe  0xe0241bfe  0xe0301bfe  0xe0311bfe  0xe0321bfe  0xe0331bfe  0xadeadcdf</pre> 
		Data for e022 is f9adf8ad f9adfaad fbadf8d0.  Cannot make heads or tails of it.  Sometimes it is 00000000 instead. 
	      <li> Fib ids for subset SB0W8 L1-L4 and SB0W9L0-L4 just something in data stream <B>Total: 102</B> 
		<pre>FIB ids found: 
  0xe0800cfe  0x6f087007  0x1d041e06  0xadeadcdf  0x4bf34c08  0x4ff35008  0x3f134014  0x45f34f01  0x16031706  0x3cf33d06</pre> 
		Instead of e0810cfe you get 6f087007 71097204 78f3794a which is just some data.  It would be very nice 
		to see the whole event to see if the pointers are screwed up (although this check is done).  In this case<ul> 
		  <li> There was just previously  an error for this Vrb on the SCPU: 
		    <pre>(MLE) b0l3gate1.fnal.gov:main:2:23:25 AM->Host b0eb17.fnal.gov, task tRec_0 
SCPU-P0-E-BadVrbByteCount: Event 2255 total byte count is lt 32, gt 65528, or not divisible by 8 for VRB in slot 18.Count = 3315. 
(MLE) b0l3gate1.fnal.gov:main:2:23:25 AM->Host b0eb17.fnal.gov, task tRec_0 
SCPU-P0-E-VrbHeader: Dump of header words for event 2255 from VRB in slot 18: 
0x0cf34bf3 0x4c0b4d09 0x4e144f18 0x500a5107 0x520166f3 0x670c681b 0x69176a01 0x7ff3c1c1</pre> 
which looks to me like data instead of a VRB header.  This may be the root cause. 
		  <li>Just after, we get a BUSY timeout on that VRB.   
		</ul> 
		This is not always the case, however. 
	      <li> What should be Fib Ids look like data-Location Unknown <B>Total: 4</B><BR> 
		The only notable thing is that there were no complaints about the Byte count sums 
		or anything 
	      <li> Problem with top nibble of FIB/VRB Chan 9 - the Split HDI! <B>Total: 9 </B><BR> 
		This is when the top nibble of the byte in the split channel is incorrect, indicating some 
		glink problem.  Distribution is: 
		<table> 
		  <tr> <th> Ladder(s) </th> <th> # Occurances </th> </tr> 
		  <tr> <td> SB3W7 L4 </td> <td> 4 </td> </tr> 
		  <tr> <td> SB1W1 L4 </td> <td> 2 </td> </tr> 
		  <tr> <td> SB3W3 L4 </td> <td> 1 </td> </tr> 
		  <tr> <td> SB4W3 L4 </td> <td> 1 </td> </tr> 
		  <tr> <td> IB1W5 L4 </td> <td> 1 </td> </tr> 
		</table> 
	    </ul> 
	  </ol> 
      <li> <h4>DLINCO: Byte Count != Sum- Total: 2685 </h4>  
      This is when the Total Byte Count for a VRB does not equal the sum of its channels. 
      This is strange because the check is also done on the SCPU, so an error in 
      the reformatter without an error in the SCPU indicates a problem between them, in  
      the ATM switch.  There is a known problem of the type in b0eb17, where at least 2625 of these 
      errors come from. <BR> 
      The other time you get this error is when the total bytes overflow the 16 bit counter. 
      This can happen anytime you get the infamous 8KB of 00f3 on 8 or more channels in the same  
      event, and this happens.  Finally, running with the SEVB the check is not made 
      on the crate, and it seems lately we have seen things get out of synch a little, though 
      the debug makes no sense either (See run 137433) 
      <li> <h4>BNK_INSTAN: Imposter FIB ID- Total: 17583 </h4> 
      This is when the RF receives some data from a ladder it has already seen once in this event.  Presumably a Fib is 
      not setting its ID correctly.  The only thing it reports is what the "instance" is, where "instance" is  
      calculated by an algorithm designed to turn the fib id into a number between (1 and nLadders) for SVX and (0 and NLADDERS-1) 
      for ISL.  By decoding this instance, one can find out who the imposter is pretending to be (but we need to add some 
      more debugging to the RF to see if we can find out who the imposter actually is...).  Distribution of false Fib Ids: 
		<table> 
		  <tr> <th>instance</th> <th> Ladder </th> <th> # Occurances </th> </tr> 
		  <tr>  <td> 1 </td> <td> SB0W0 L0 </td> <td> 4207 </td> </tr> 
		  <tr>  <td> 11 </td> <td> SB0W2 L0 </td> <td> 4 </td> </tr> 
		  <tr>  <td> 71 </td> <td> SB1W2 L0 </td> <td> 2 </td> </tr> 
		  <tr>  <td> 81 </td> <td> SB1W4 L0 </td> <td> 1 </td> </tr> 
		  <tr>  <td> 121 </td> <td> SB2W0 L0 </td> <td> 2783 </td> </tr> 
		  <tr>  <td> 151 </td> <td> SB2W6 L0 </td> <td> 462 </td> </tr> 
		  <tr>  <td> 191 </td> <td> SB3W2 L0 </td> <td> 6276 </td> </tr> 
		  <tr>  <td> 241 </td> <td> SB4W0 L0 </td> <td> 15 </td> </tr> 
		  <tr>  <td> 251 </td> <td> SB4W2 L0 </td> <td> 2 </td> </tr> 
		  <tr>  <td> 261 </td> <td> SB4W4 L0 </td> <td> 2 </td> </tr> 
		  <tr>  <td> 10 </td> <td> IB0W2 L0 </td> <td> 183 </td> </tr> 
		  <tr>  <td> 20 </td> <td> IB0W4 L0 </td> <td> 20 </td> </tr> 
		  <tr>  <td> 50 </td> <td> IB1W4 L0 </td> <td> 1 </td> </tr> 
		</table> 
      <li> <h4>STAICO: Status Word Format- Total: 23 </h4> 
    There are two Status words in the Vrb header, whose lowest 2 bits should be 0.  When they're not, the RF 
    complains.  I believe the check is done before the total byte count check, because usually it means 
    what is thought to be status words actually aren't. 
      <li> <h4>VRBWRD: Bad TSL1/TS Event Number- Total: 55 </h4> This is  
    when the number the Vrbs receive from the SRC for the "Time Since L1A" and "TS Event Number" do not match up 
    within one event.  This time it came from two runs<ul> 
      <li> 137340: 3 times, when the TS # was ok but not TSL1A 
      <li> 137640: The rest, when 0x8000 was expected but 0x7e02 was found.  This can happen when 
	one does a HRR with the SEVB, probably the case here. 
    </ul> 
  </ul> 
 
    <A NAME = "theEndofTheEntry">The End</A>       
   
<!-- Author: SCN -->