<!-- Title: 10:44:31  Mon Nov 19 2007 -->
<!-- Crewchief: SashaS (MPC) -->
<!-- Op1: Ricardo (SPL) -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<ul> 
<li>Keep Silicon in healthy operation. 
<li>Continue investigation of SRV/Clock errors 
<li>Observe situation with losses: go in with Silicon if Lostp greater than 10kHz (current warning limit) and everything else is Ok 
</ul>

<!-- Date: Mon Nov 19 14:01:47 2007 -->
The pH  
 reading by KC this morning was  
<pre> 
Conductivity for SVX is .3µS and ISL is .4µS. pH for SVX is 4.99 and ISL is 5.31. 
</pre> 
the system meter was 0.65 and 0.47, so it seems the [0.6,0.5] range   for the controller would work OK.
<!-- Author: Ignacio -->


<!-- Date: Mon Nov 19 14:06:47 2007 -->
CDF was switch back to its original feeder this morning 10:30. Nothing was noticed.
<!-- Author: Ignacio -->


<!-- Date: Mon Nov 19 14:14:52 2007 -->
Changed IMON limits for BIAS0/BIAS1 currents: set variance to 0.2 and minimum to 0.5 for the whole detector (SVX/ISL/L00). This will help us to catch ladders with 0 bias current.
<!-- Author: Jose/SashaS -->


<!-- Date: Mon Nov 19 14:22:08 2007 -->
We decided in the meeting this morning that for the next week we will keep the TevMon ALARM limits, i.e. we will not bypass TevMon.  If there is a warning due to high losses at the beginnig of the store, we will ask  MCR to check the position of the collimators but not to try to adjust the tunes. 
Since the losses should go down after the fist ~hour, we will go in  straigh ahead or as soon MCR confirms that everything is normal. The MPC should stay until the warning clears. 
  The stores this weeks will be used to review  the WARNING and ALARM limits.
<!-- Author: Ricardo and  Ignacio -->


<!-- Date: Mon Nov 19 15:33:25 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12055&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12055  width='300' align='top'></a></td><td>As unmarking all cells in IMON did not show any pinkies, I plot the Abort Plot.</td></tr></table>
<!-- Author: Adrian -->


<!-- Date: Mon Nov 19 16:10:55 2007 -->
<h3>Week Summary template</h3> 
For the convenience of the MPC, you can find a template to ease the writing of the weekly summary in  
<pre> 
b0sili01~svxii/MPCsetup/week_sum.sh  
</pre> 
Executing ./week_sum.sh  will ask you for the date of the meeting and will oputput /week_sum.html with the correct dates in the template.  
<!-- Author: Ignacio -->


<!-- Date: Mon Nov 19 18:44:21 2007 -->
<p>We have setup LA on 1st floor connected to the TRACER in an empty VME create (1RR27C/b0c3po, just below b0imu01). We have connected CDF_CLK, B0, BC, BG signals to LA and setup external trigger from the Silicon LA to trigger on SRC error signal. We will check if we will see any clock errors in this LA correlated with SRC errors.
<!-- Author: Sebastian/Jose/Ricardo/SashaS -->


<!-- Date: Tue Nov 20 08:31:19 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12056&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12056  width='300' align='top'></a></td><td>Relative small clk glitch in both svx and isl.</td></tr></table>
<!-- Author: Sebastian -->
<!-- Comment: Tue Nov 20 08;39;50 comment by...Sebastian -->
<pre>The glitch is about 0.5us long. This happened around 6:30 (run 254013). The tracer LA did trigger on the ext trigger from the silicon LA but no glitch is seen there. </pre>

<!-- Comment: Tue Nov 20 08;42;12 comment by...Sebastian -->
Trace saved as clk_glitch_20nov07.ala in the silicon LA.

<!-- Comment: Tue Nov 20 09;04;17 comment by...Ignacio -->
Times to understand the glitch:
<pre>
06:30:17 Config -> READY (Error Logger)
06:32:17 SRC error (triggers LA time from e-mail)
06:35:48 Ladders trip (PS GUI)
06:38:24 AM->SRC Fatal Error:Sl 6 2 L1A to Buff (Error Logger)
</pre>
What was the RC doing between 6:32:17 and 6:38:24? I think the SRC swas in a strange state after the glitch, 3 minutes without sending a CONFIG or a RECOVER was enought to trip the ladders.


<!-- Comment: Tue Nov 20 14;45;47 comment by...Ricardo -->
Is it clear that a recover would have worked ?. <br>
To see a more detail time sequence see <a href=
'http://www-cdfonline.fnal.gov/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=380&frame=2&anchor=14:23:15'>
here.</a> 



<!-- Date: Tue Nov 20 10:12:39 2007 -->
We will switch off L00 for the separator scan: 
<pre> 
The plan: drop the store late evening, separator scans later this store (this afternoon?) - Phil 
</pre>
<!-- Author: Ignacio -->


<!-- Date: Tue Nov 20 11:01:31 2007 -->
<h3>Pinkies:</h3> 
ISL <b>IB2W2L3 IBias1</b> and <b>IB4W1L1 IBias1</b> currents read back 0. Data (taking into account sparsed chips in these ladders) seems ok in SVXmon. <b>Need to hockerize PS 7 and PS 16</b> in between stores.
<!-- Author: Ignacio/SashaS -->


<!-- Date: Tue Nov 20 11:07:55 2007 -->
<h3>Removed from HWDB: Ladder ISL IB4W0L4</h3> 
We could not power IB4W0L4 ladder this morning during recovery of massive trip. It tripped on DVDD ovc. Had to remove it from HWDB. 
There is entry in PDB by Ignacio: 
<pre> 
This ladder does not trip any more on DVDD. The change came by setting the rdriver of the first chip from 5 to 3 from svxdaq. After that, the ladder would not trip regardless of the rdriver setting. I run 10k from coldstart without problem. I leave the ladder in but we should take it out again in case it trips again. There are 00 sequence errors when running from svxdaq, but I want to see what happens in real data taking. 
</pre> 
We will work on this in between stores
<!-- Author: Ignacio/SashaS -->


<!-- Date: Tue Nov 20 13:20:47 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12057&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12058  width='300' align='top'></a></td><td>In the runs taken lately (weekend) the underdepleted ladders show a physics tail. It is not great, but it was like that before the crowbars failed. The plot shows fa03... it is the same for the other. </td></tr></table>
<!-- Author: Oscar -->

<!-- Date: Tue Nov 20 14:20:42 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: IB4W0L4 
<li>problem n: 2173 
<li>Status: needs_work
<li>Type: DVDD
<li>Action: Removed_hdwdb
<li>entered by: sukhanov 
<li>description is: Ladder trips on DVDD ovc. Removed from HWDB 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=ISL&wedge=B4W0&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Tue Nov 20 14:23:15 2007 -->
This is the sequence of events that didn;t allow us to put tht silicon in the store right away. <br> 
Time Sequence:<br> 
 
<ul> 
<li>06:28:02 to 06:30:03 CCC reports a clock glitch.</li> 
 
<li>06:30:17 (RC)  6:30:17 Config -> READY</li> 
 
<li>06:32:01 to 06:36:02 CCC reports a clock glitch.</li> 
 
<li>06:32:17 SRC reports error via email- All fibs are screwed at this point.</li> 
 
<li>06:33:10 Ace start biasing up.svxinter gets busy.</li> 
 
<li>06:35:48 Powering up fails due to handshake with screwed fibs. IFIX-error on handshake. </li> 
 
<li>06:38:24 SRC errors are reported only after b0svx02 and b0svx06 crates finished config.</li> 
 
<li>06:42:07 Ace goes to Reset directly (no abort done first)</li> 
</ul> 
 
 
Basically a clock glitch during in the early stages of starting the run. Had this happened during data taking RC would have sent RC to halt on the SRC error, and the ace  would have click on recover right away, but would that have "repaired" the fibs ?
<!-- Author: Ricardo & SashaS -->



<!-- Date: Tue Nov 20 15:00:02 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12063&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12064  width='300' align='top'></a></td><td>Congratulations Saty!!   
A potential silicon member was born ! 
</td></tr></table>
<!-- Author: Ricardo -->


<!-- Date: Tue Nov 20 15:26:53 2007 -->
The DS links were all looking  to http://hepdhcpd.physics.ucsb.edu 
rather than http://stuart.physics.ucsb.edu. I have changed  them everywhere I have found them: PDB, BitReporter, Inefficiency reporter. The owner of  the inefficency reporter is svxii now.
<!-- Author: Ignacio -->

<!-- Date: Tue Nov 20 17:12:12 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB0W0L2 
<li>problem n: 1490 
<li>Status: needs_work
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Wrong CAP_ID on chip 89 in ~90% of events starting from 11/06/07. As suggested in the previous entry, chip 89 can be sparsed out (though it's not going to cure CAP_ID) if Monitoring group confirms that data from this chip is useless. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B0W0&layer=L2">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:16:22 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB0W3L3 
<li>problem n: 1514 
<li>Status: fixed
<li>Type: BIAS
<li>Action: Monitoring
<li>entered by: sasha 
<li>description is: This is the same thread as 1556. I close this one. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B0W3&layer=L3">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:19:40 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB0W5L2 
<li>problem n: 1829 
<li>Status: needs_observation
<li>Type: DRO_00F3
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: The error rate is moderate (~20%). Need to continue observations and keep in mind solution suggested earlier if rate goes to >70-80%. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B0W5&layer=L2">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:23:10 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB0W10L4 
<li>problem n: 2064 
<li>Status: fixed
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: No errors in data. Fixed. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B0W10&layer=L4">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:25:15 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB1W1L4 
<li>problem n: 2042 
<li>Status: fixed
<li>Type: Other
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Ladder looks OK in BitReporter and ADCMon. Fixed. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B1W1&layer=L4">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:28:47 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB1W9L0 
<li>problem n: 2111 
<li>Status: fixed
<li>Type: BIAS
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Closing this thread. We usually put IMON_limits into 'Fixed' category. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B1W9&layer=L0">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:37:44 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W7L2 
<li>problem n: 1387 
<li>Status: unfixable
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: No light at FTM. Declaring it unfixable. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B2W7&layer=L2">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Tue Nov 20 17:39:17 2007 -->
<ul> 
<li>Have put Silicon to STBY and turned off L00 for the MCR separator study. 
<ul> 
<li>We had problem with the standard (green font) <i>svxinter</i> termial: it seemed to be forzen and did not respond to any input. We vxlogin to <i>svxinter</i> from the other terminal window and rebooted <i>svxinter</i> CPU, but the standard terminal remained frozed. We killed terminal and started new one. 
<li>Next time, if it will appear that <i>svxinter</i> frozen, try to vxlogin  to the crate and check status of processes. It might not require to reboot the crate, but just restart terminal window. 
</ul> 
<li>ASA MCR was done with bias scan we put L00 to STBY. 
</ul>
<!-- Author: Ignacio /SashaS -->

<!-- Date: Tue Nov 20 17:40:22 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W11L2 
<li>problem n: 1993 
<li>Status: fixed
<li>Type: TX
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: No errors for past 30 days. Fixed. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B2W11&layer=L2">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:43:54 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB3W2L0 
<li>problem n: 868 
<li>Status: needs_observation
<li>Type: Chip_capid
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Error rate is ~10%. We can live with this, given that our previous history of adjusting BE kludge is not so bright. Continue observations. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B3W2&layer=L0">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:50:08 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB3W9L3 
<li>problem n: 1410 
<li>Status: unfixable
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Errors in data are consistent with dead bit3, as it was measured twice in the past. Nothing can be done for this problem. Unfixable.  
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B3W9&layer=L3">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:52:13 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W1L4 
<li>problem n: 1875 
<li>Status: fixed
<li>Type: DVDD
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: No other reports of this problem. Fixed. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B4W1&layer=L4">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 17:54:07 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W2L4 
<li>problem n: 1873 
<li>Status: fixed
<li>Type: BIAS
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Closing this thread because adjustment of IMON limits usually gets reported under category 'Fixed'. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B4W2&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Tue Nov 20 17:54:08 2007 -->
A quadrant of a COT superlayer is off. They masked the inhibits and the run is going to be marked bad for COT. Is is expected that the trigger rates will be high, so a new run will be made without silicon and we'll check the trigger rates after the prescale kicked in.  We might consider include the silicon after these tests are done.  
<!-- Author: SashaS and Ricardo -->
<!-- Comment: Tue Nov 20 18;08;48 comment by...Ricardo & Sasha -->
Since the COT is going to be marked off, and the data thrown away, I don't see the point of including Silicon unless the COT problem is fixed, or we wanted to make some tests. Let's talk to operations about this.

<!-- Comment: Tue Nov 20 18;16;35 comment by...SashaS & Ricardo -->
<table border=0><td><tr><td bgcolor=#e0e030>Due to potential problems with the trigger rates, and the fact that data will be marked bad for COT, we decided to leave the silicon out until the COT problem is fixed. We might still put Silicon in to do tests related to the current clock glitches. If anyone has a good reason to run with the silicon speak now, or forever...</td></tr></td></table>
<!-- Comment: Wed Nov 21 10;43;47 comment by...Ricardo -->
In the 8 oclock meeting of this morning Bob Wagner (COT) said he thinks we could in principle recover data taken with the current COT configuration. Since we have no intention of keeping the silicon out if there is potential to recover the data, we can put the silicon provided there is no other concern like trigger rates, etc. Let's make sure the trigger rates are OK, specially in the luminosity range where the prescales kick in, and put silicon in if all other conditions are OK. 
<!-- Date: Tue Nov 20 17:58:17 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W9L4 
<li>problem n: 2041 
<li>Status: fixed
<li>Type: Other
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Ladder looks OK in both BitReporter and ADCMon. Fixed. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B4W9&layer=L4">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 18:02:27 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB5W6L4 
<li>problem n: 345 
<li>Status: unfixable
<li>Type: Optical_other
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Errors in data are consistent with bit 6,7 low (as was measured in the past). Not much can be done. Unfixable. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B5W6&layer=L4">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Tue Nov 20 18:12:50 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W2L3 
<li>problem n: 436 
<li>Status: needs_work
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Bit5 low is back. Need quiet time studies. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B2W2&layer=L3">report</a></ul>
<!-- Author: Problem DB -->


<!-- Date: Tue Nov 20 18:15:00 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB2W2L4 
<li>problem n: 1423 
<li>Status: needs_work
<li>Type: Chip_bit_error
<li>Action: Diagnosis_update
<li>entered by: sasha 
<li>description is: Problems with chip89 are back. Needs quiet time studies. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B2W2&layer=L4">report</a></ul>
<!-- Author: Problem DB -->




<!-- Date: Tue Nov 20 18:18:10 2007 -->
<pre>I did PDB and BitReporter review for all SVX ladders. I only looked for problems with chips (didn't check PS related threads). Hugo promissed to do the same for L00 and ISL. </pre>
<!-- Author: SashaP -->


<!-- Date: Wed Nov 21 09:40:27 2007 -->
Bill noe called to arrange a time to see (or rather try to)  reboot the svxiicon. This was done only a week ago, but we took the oportunity and Ulrich showed him how it is done. He will make a procedure for his techs to follow.
<!-- Author: Ricardo & Ulrich -->




<!-- Date: Wed Nov 21 17:01:37 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12065&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12065  width='300' align='top'></a></td><td>Store 5741: np ~7800, npb ~2500, Init L ~180, losses ~8kHz/6kHz; trigger rates L1A ~9kHz. Decided to put Silicon in.</td></tr></table>
<!-- Author: SashaS -->


<!-- Date: Wed Nov 21 17:59:06 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12066&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12066  width='300' align='top'></a></td><td>Where did the clock go?</td></tr></table>
<!-- Author: SashaS/Ricardo/Jose/Ignacio -->
<!-- Comment: Wed Nov 21 18;00;45 comment by... the ace -->
Wed Nov 21 17:58:12  BMU tripped in SE wedge 156

<!-- Comment: Wed Nov 21 19;09;06 comment by...Ricardo -->
Just to be prepared if this happens again I set the scope in the first floor to trigger on the Sync clock signal if any event has a positive width of less than 104 ns or more than than 126 ns. This is +/- 10 ns from its average size of 115.
 

<!-- Date: Wed Nov 21 19:33:41 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB5W6L4 
<li>problem n: 2462 
<li>Status: svxmon_ignore
<li>Type: Other
<li>Action: Monitoring
<li>entered by: jgarcian 
<li>description is: This ladder is giving no data, set to ignore for SVXMon by now. 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B5W6&layer=L4">report</a></ul>
<!-- Author: Problem DB -->
<!-- Comment: Wed Nov 21 20;23;39 comment by...Jose -->
It is not completely true, it was giving some chips were giving some data but was almost empty.
<!-- Date: Wed Nov 21 19:49:25 2007 -->
Entered in Silicon database: <ul>
<li>Ladder: SB4W6L0 
<li>problem n: 1735 
<li>Status: needs_observation
<li>Type: DVDD
<li>Action: IMON_limits
<li>entered by: ignacio 
<li>description is: The DVDD current has been slowly increasing.<table border=1>
<tr><th>IMON LIMITS</th><th>Old</th><th>New</th></tr>
<tr><td><b>Nominal</b></td><td>81</td><td>90</td></tr>
<tr><td><b>Lower</b></td><td>50</td><td>50</td></tr>
<tr><td><b>Upper</b></td><td>85.0</td><td>100.0</td></tr></table> 
<li>See <a href="http://b0sili01.fnal.gov/cgi-bin/pdb/DBreport.pl?detsys=SVX&wedge=B4W6&layer=L0">report</a></ul>
<!-- Author: Problem DB -->



<!-- Date: Wed Nov 21 19:52:38 2007 -->
There were 4 bias pinkies after the UPS kicked in at 110E30. L1A was 27.5 KHz. I ignore them for the moment.
<!-- Author: Ignacio -->



<!-- Date: Wed Nov 21 20:59:16 2007 -->
Got an unnecessary page because of oscillations in the O2 monitoring for the ISL vacuum pump exhaust. I strongly suggest to take this variable off the control room real time plotter or leave clear instructions to the shift crew what to actually monitor.
<!-- Author: Ulrich -->
<!-- Comment: Thu Nov 22 13;40;21 comment by...Ignacio -->
Took out the O2 reading (Fix32.CDF_S2.ISL_VAC_O2.F_CV) from the RealTimePlotter.
<!-- Comment: Thu Nov 22 13;49;41 comment by...Ricardo -->
We also told the aces not to react to the ifix real time plotter. For serious conditions we have the IFIX alarms.
<!-- Date: Thu Nov 22 05:03:36 2007 -->
Got page (1st to AN from L00/ISL PS, then to main pager from CR) for spantaneous switch off of LB0W0, LB0W5, IB0W4, IB0W5, IB2W2 wedges. All wedges correspond to fib03 crate. Ace was succesfull in turning on LB0W5, IB0W4, IB0W5, IB2W2 but LB0W0L3 tripped on Dvdd ovc. Ace also mentioned Bias trip ovc but I could not find error message on that in PSGui log. Here is <i>svxinter</i> messagges: 
<pre> 
svxinter-> THU NOV 22 03:11:56 2007  Caenet_buffer: Error number 7 7 received 
group_info:cr_n = 7, response = 7 
THU NOV 22 03:12:05 2007  Caenet_write: Error number -251 ffffff05 received 
group_info:cr_n = 7, response = -251 
THU NOV 22 03:12:16 2007  Caenet_write: Error number -1 ffffffff received 
group_info:cr_n = 7, response = -1 
THU NOV 22 03:12:24 2007  Caenet_buffer: Error number 7 7 received 
group_info:cr_n = 7, response = 7 
THU NOV 22 03:15:33 2007  setPSStatus, detector = 0, ps_n = 0, status = 0 
THU NOV 22 03:15:33 2007 Sending Request for f806 
THU NOV 22 03:15:36 2007 PS is still not ON, after 10 checks 
</pre> 
and messages from <i>PSGui</i> log: 
<pre> 
Thu Nov 22 03:15:36 CST 2007                        ERROR: Power Supply L00 WEST W0 Handshake - Could not turn on power - checked 10 times 
Thu Nov 22 03:15:39 CST 2007                        TRIP: L00  WEST  W0 DVDD3 OverCurrent 
</pre> 
So it looks like lost communication to fib03 crate. I turned off by fib03, rebooted fib03 and turn on by fib03. L0W1L3, L0W4L3 tripped Dvdd ovc, had to turn them by wedge. 
<pre> 
Thu Nov 22 03:58:27 CST 2007                        Turning b0fib03 to All off 
Thu Nov 22 04:02:17 CST 2007                        Turning b0fib03 to All standby 
Thu Nov 22 04:02:49 CST 2007                        TRIP: L00  WEST  W1 DVDD3 OverCurrent 
Thu Nov 22 04:02:49 CST 2007                        TRIP: L00  WEST  W4 DVDD3 OverCurrent 
Thu Nov 22 04:02:49 CST 2007                        ERROR in setFibStatus.  PAY ATTENTION, this is a problem.  At least one CAEN power supply failed to respond correctly.  Received 1 errors. 
Thu Nov 22 04:03:20 CST 2007                        success: Power Supply L00 WEST W4 OFF 
Thu Nov 22 04:03:28 CST 2007                        success: Port Card L00 WEST W4 OFF 
Thu Nov 22 04:04:06 CST 2007                        success: Power Supply L00 WEST W4 to STANDBY 
Thu Nov 22 04:04:44 CST 2007                        success: Port Card L00 WEST W1 OFF 
Thu Nov 22 04:04:52 CST 2007                        success: Power Supply L00 WEST W1 to STANDBY 
</pre> 
After everything was to STBY, asked ace to put Silicon in.
<!-- Author: SashaS -->


<!-- Date: Thu Nov 22 05:15:14 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12068&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12068  width='300' align='top'></a></td><td>B0AAGC losses have spikes up to ~10 kHz. Mean value is about 2kHz and RMS 1.5kHz. These are below warning limits (10+-2)kHz in Tevmon but I asked ace to keep eye on these and page me if will be at warning state</td></tr></table>
<!-- Author: SashaS -->


<!-- Date: Thu Nov 22 05:28:22 2007 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12069&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12069' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12070&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12070' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12071&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12071' width='300'></a></center>
<!-- Author: SashaS -->
<!-- Comment: Thu Nov 22 05;33;04 comment by...SashaS -->
<p>Compared AAGC from stores 5729, 5730 and 5731. In stores 5729 and 5730 losses are a bit lower, in store 5731 AAGC comparable to that from the current store 5741.


<!-- Date: Thu Nov 22 05:41:50 2007 -->
<pre>Auto-hockerization of PS6 (fib02). Recovered successfully</pre>
<!-- Author: SashaS -->


<!-- Date: Thu Nov 22 05:47:18 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12072&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12072  width='300' align='top'></a></td><td>SVTPSYmon plot after recovering spontaneous switch-off of fib03</td></tr></table>
<!-- Author: SashaS -->




<!-- Date: Thu Nov 22 13:49:58 2007 -->
<table border=0><td><tr><td bgcolor=#e0e030>My e-mail stopped accepting all messanges  yesterday 10:30 AM. It had dropped some of them in the previous  days. You can contact me at ignacio.redondo@ciemat.es  or use  my cell.</td></tr></td></table>
<!-- Author: Ignacio -->


<!-- Date: Thu Nov 22 16:05:25 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12073&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12073  width='300' align='top'></a></td><td>Store 5743: Stash 300; np ~8500, npb ~2500; L = 175; Lostp ~16kHz, Lostpb 5.5kHz; L1A ~8.5kHz, L2A ~250Hz. Took short run w/o Silicon to monitor losses and trigger rates. Losses were smooth and going down, trigger rates ok. Decided to put Silicon in.</td></tr></table>
<!-- Author: SashaS -->
<!-- Comment: Thu Nov 22 16;15;28 comment by...SashaS -->
Tevmon is clear now: Lostp below 10kHz


<!-- Date: Thu Nov 22 23:03:35 2007 -->
While Aces were doing test of the new trigger table w/o Silicon (but Silicon is still biased) I  noticed fib01 turned red in <i>VxWorks System Monitor</i>. I did COLDSTART and found heap corrupt error in fib01 crate. Rebooted crate. Everything is ok now
<!-- Author: SashaS -->


<!-- Date: Thu Nov 22 23:15:46 2007 -->
<p>At about 10:06pm I have got AN page on SVX/ISL SMTH OFF. Called back to the CDF Control Room. They have auto-hockerization of PS 10. Recovered affected SVX/ISL wedges back without problem. Great job done by Donatella and Sam, today day shift aces.
<!-- Author: SashaS -->


<!-- Date: Thu Nov 22 23:34:40 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12074&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12074  width='300' align='top'></a></td><td>Test of the new trigger table was un-successful due to L2 executable crashes. Turned back to RunSet: AAA_CURRENT_L2UP Trigger: PHYSICS_5_01 [2,1002,710]</td></tr></table>
<!-- Author: SashaS -->


<!-- Date: Fri Nov 23 00:06:16 2007 -->
<p><b>b0svx02</b> error, svx02 crate turned red in RC client window. Then followed <i>SRC Fatal Error from b0svx02: Sl 6 Resonance Detected</i>.<br> 
Stopping run and starting new run cleared the problem  
<!-- Author: Sasha -->


<!-- Date: Fri Nov 23 06:44:52 2007 -->
<p>SB3W10L4 tripped on Dvdd ovc when North wall moved. It must be the same ladder which always trip when the N wall moves
<!-- Author: SashaS -->


<!-- Date: Fri Nov 23 07:23:43 2007 -->
Silicon is OFF for access.  
<!-- Author: Ricardo -->
<!-- Comment: Fri Nov 23 07;26;51 comment by...SashaS -->
Set Auto-Hockerization to <i>MANUAL</i>


<!-- Comment: Fri Nov 23 08;01;08 comment by...Ricardo  -->
5V was disable
<!-- Comment: Fri Nov 23 08;03;55 comment by...Ricardo -->
IMON was stopped and checks were disabled in Procmon.

<!-- Comment: Fri Nov 23 19;35;46 comment by...Ricardo -->
Auto-hocker is back to auto, 5V back online and checks are enabled back in ProcMon 
<!-- Date: Fri Nov 23 07:51:29 2007 -->
HV for Diamonds and BLMS are off and locked up. Since Dervin was busy Ulrich and I took them off.
<!-- Author: Ricardo & Ulrich -->




<!-- Date: Fri Nov 23 10:54:05 2007 -->
COT baggie was slit. The ace and the cooling group is keeping an eye on dew points.
<!-- Author: Ricardo -->


<!-- Date: Fri Nov 23 11:12:37 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12075&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12075  width='300' align='top'></a></td><td>We connected a LA signal to the CDF SYNC input in U31, so we now can look directly at what goes into the SRC.</td></tr></table>
<!-- Author: sasha/sebastian -->
<!-- Comment: Fri Nov 23 11;13;20 comment by...s/s -->
The red trace is CDF SYNC, (also shows as MCLKSVX6).


<!-- Date: Fri Nov 23 12:55:27 2007 -->
The baggie is up. They have started with Proc 13, we are cooling down ISL to 10 C, SVX to -6 C.  
<!-- Author: Ignacio -->
<!-- Comment: Fri Nov 23 13;50;16 comment by...Ignacio -->
Talked to Del, we are on scheadule. We should be able to start checkout from 14:00. (they will infrom the CR)


<!-- Date: Fri Nov 23 15:59:55 2007 -->
<h3>Check-out</h3> 
<h4>ISL and L00 are OK</h4> 
<br>After the end of the COT access, we have run SVXDAQ to check ISL and L00 and found that everything is as good as expected (consistent with the PDB).<br>
<!-- Author: Jose / Miguel M. -->
<!-- Comment: Fri Nov 23 16;06;59 comment by...Ignacio / Sasha S -->
<h4>SVX is OK</h4>
SVX has been also checked.

<!-- Comment: Fri Nov 23 16;54;00 comment by...Tom & Ulrich -->
The cooling system is back to default: SVX at -10°C, ISL at +6°C. There are just a couple of alarms to be unsilenced in an hour or so. We also did the following tweaks:
<ul>
<li>After the shutdown we forgot to reset the SVX BH and L00 PLC temperature trip limits from +6°C to +2°C. We did that now.
<li>We realized that the SVX vacuum pump (VP1A) was only capable of 1.9 psia. We switched to VP2-A, and the vacuum improved to 1.4 psia or better. We will ask Bill/KC to rebuild the pump.
</ul>



<!-- Date: Fri Nov 23 19:36:57 2007 -->
Beam Permit was killed until we finish running cosmics.  
<!-- Author: Ricardo -->
<!-- Comment: Fri Nov 23 20;05;36 comment by...Ricardo -->
Cosmics run is over after about an hour. I have put the BeamPermit and made sure conditions are clear for data taking.


<!-- Date: Sat Nov 24 03:18:50 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12076&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12076  width='300' align='top'></a></td><td>Store 5745 was declared by 2am with Init. Lumi. ~170, Lostp ~27kHz, Lostpb ~5.5kHz. Started w/o Silicon; COT tripped in SL7/8, Aseet was working on recovering COT trips. When COT was ready, losses dropped Lostp ~10kHz, Lostpb ~5kHz; L ~120; L1A ~5kHz. Decided to put Silicon in.</td></tr></table>
<!-- Author: SashaS -->
<!-- Comment: Sat Nov 24 03;50;31 comment by...SashaS -->
Lost beam at 3:30 - not a quench


<!-- Date: Sat Nov 24 06:18:32 2007 -->
I have got page because there is a message on the white board in the control room to page silicon if there is >2h quite time. Does anyone know if there is anything special MPC is suppose to do during long quite time?
<!-- Author: Sasha -->
<!-- Comment: Sat Nov 24 07;19;13 comment by...Ulrich -->
Quiet time studies (if there are any pending).

<!-- Comment: Sat Nov 24 09;57;37 comment by...SashaS -->
Ok, next time I will come

<!-- Comment: Sat Nov 24 10;42;06 comment by...Jose -->
That was probably there from the end of the shutdown. We wanted a lot of quiet time for cosmics.


<!-- Date: Sat Nov 24 14:46:27 2007 -->
Store 5747 has been lost at flattop. Abort pulled due to D0 SVX losses.
<!-- Author: SashaS -->
<!-- Comment: Sat Nov 24 14;48;18 comment by...SashaS -->
I have done <i>COLDSTART</i> (walked ace through) as part of post-quench procedure. Silicon crates are ok


<!-- Date: Sat Nov 24 18:08:29 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12077&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12077  width='300' align='top'></a></td><td><h3> Losses at D0 </h3> 
No surprise they pulled the abort.</td></tr></table>
<!-- Author: Jose -->


<!-- Date: Sun Nov 25 00:37:30 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12078&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12078  width='300' align='top'></a></td><td>Store 5748: Np ~7300, ~200/bunch; Npb ~1200, ~30/bunch; Init. lumi ~114; Lostp ~11kHz, Lostpb ~4kHz; Trigger rates looked normal, decided to put Silicon in</td></tr></table>
<!-- Author: SashaS -->


<!-- Date: Sun Nov 25 00:41:17 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12079&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12079  width='300' align='top'></a></td><td>Lostp have some structure. Will keep eye on those</td></tr></table>
<!-- Author: SashaS -->


<!-- Date: Sun Nov 25 02:57:39 2007 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12080&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12080' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12081&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12081' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12082&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12082' width='300'></a></center>
<!-- Author: SashaS -->
<!-- Comment: Sun Nov 25 03;01;18 comment by...Sasha -->
We had 3 resonances in row, Run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254300>254300</A>. Took Silicon out; put it to STBY; paged L1

<!-- Comment: Sun Nov 25 03;15;04 comment by...Sasha -->
No responce from L1 trigger people. Since luminosity getting lower, we will put Silicon in.

<!-- Comment: Sun Nov 25 03;37;23 comment by...Sasha -->
Happened again, took silicon out, put it to STBY

<!-- Comment: Sun Nov 25 05;40;23 comment by...Sasha -->
We can not find any particular problem with triggers. Resonances seem not to be correlated with PS changes - happened from 5 to 20 min after last PS change. Decided to put Silicon in and observe.

<!-- Comment: Sun Nov 25 05;41;48 comment by...Sasha -->
Two more resonances occured at 4:30 and 4:55. After the last one stoped and re-started run.

<!-- Comment: Sun Nov 25 06;00;17 comment by...sasha -->
No resonances since we restarted run ~1h ago. I will left it running.
<!-- Comment: Sun Nov 25 08;13;32 comment by...Ignacio -->
Resonances started after the UPS kicked in. The XFT spike could be a source (Nils).


<!-- Comment: Sun Nov 25 09;15;06 comment by...Sasha -->
Few more resonances at 6:24, 6:27, 6:51. Ace took Silicon out, put it to STBY.<br>
Just talked to Nils. He said that XFT is ok and he can not see any hot triggers. Re-included Silicon, started new run <A HREF=/java/cdfdb/servlet/RunSummary?RUN_NUMBER=254315>254315</A>.
No resonances, ended run to exclude Silicon due to TeV RF station 8 trip
<!-- Comment: Sun Nov 25 09;23;46 comment by...jdl -->
The only thing the XFT could do is give hot triggers.  But that takes a strange alignment of stars or massive errors.  It's not at all like a hot calorimeter tower.  In any case, you don't have any evidence of a high L1A rate in these traces.  In fact what you see is ~15 L1A in 500us.  That corresponds to 30kHz which should be no problem.  That fact that all are rejected by L2 should also not be a problem.  With a rejection rate of 30-100, we expect that as well.  However, what does seem strange to me is that RBSY (Read-Busy) is high for ~35us on each event.  My recollection was that it takes 10us of digitize and 6-10us for readout.  Is there a chip or ladder stuck in read-all mode or some other problem?  
<!-- Comment: Sun Nov 25 10;02;13 comment by...Sebastian -->
Yes, the normal rdsy is about 20us. Since the L1/L2 rate looks fine (25kHz/400Hz in 254300) it looks like a ladder might be causing the resonances, but looking at the logs, there is no single candidate. <br>
On a side note, a few resonances in a row is something we had in the past and there is no reason to panic. We expect to get resonances, even at 25kHz.
<!-- Comment: Sun Nov 25 10;58;19 comment by...SashaS & Ricardo -->
I think Johnatan could be right, RBSY should be in average 20us, but we have seen large RBSY events in these resonances. We are investigating.
<!-- Comment: Sun Nov 25 12;13;44 comment by...Sasha -->
I would not panic much, but to(night)day we had 10 resonances in total from 2:05 to 11:21. <br>
BTW, do we have in some of Consumers a plot which shows event size distribution per vrb (or raw data bank)? Would be nice to have something like this to debug problems related to too large event.
<!-- Comment: Sun Nov 25 12;48;13 comment by...Ignacio -->
The "normal" readbusy (18 mus) peak  is always accompanied by a small 35 mus peak.  I do not understand its presence but it has been like that always (there are plots from the past in the e-log to prove it).  As SashaS and Ricardo mentions  the rate of  resonances after UPS kicked in tonight is far for normal.   The problem is that the events are garbage so they are rejected by L2 (so the consumers do not know about them). We do look at the data size   <a href='http://stuart.physics.ucsb.edu/cgi-bin/readoutocc'> offline </a>. Jonahtan, who is on charge at the new LA that is looking at the triggers?
<!-- Comment: Sun Nov 25 13;55;11 comment by...Ricardo -->
Out of the 10 resonances, 9 were pulled with RBSY of 35us length. Only one, the one at 2:23, was pulled with RBSY of 18us.  Although we have seen peaks of 35us (which I think nobody understands) i do not think these long readout events come in bursts, but rather interspread in the normal length RBSY distribution. So i tend to believe that the long RBSY has some role in these resonances, perhaps just enhancing the sensitivity to trigger bursts. 


<!-- Date: Sun Nov 25 09:09:35 2007 -->
TeV RF station trip. Put Silicon to STBY and excluded from run
<!-- Author: SashaS -->
<!-- Comment: Sun Nov 25 09;23;00 comment by...Sasha -->
Talked to MCR, they think that beam conditions are safe for. Tevmon is clean now. Decided to put Silicon in








<!-- Date: Sun Nov 25 15:24:32 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12083&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12083  width='300' align='top'></a></td><td>It's not the SRC: the trace shows a clock glitch that was caught just at the beginning of the store (see <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2007&action=view&page=920&frame=2&anchor=14:34:14.598'>here</a>). The trace shows that the input to the SRC cdf_sync is gone too (this is read directly in the differentiatior in the SRC, before any logic). </td></tr></table>
<!-- Author: sasha/sebastian -->
<!-- Comment: Mon Nov 26 09;10;23 comment by...Sebastian -->
This clk glitch was also reported by the cdfclk crate (2rr22i, PCC_CSR_6_SYNC_TIMING_ERROR), and caught in the osciloscope spying on the ouput of the clk module in the 1st floor. So, I think we should wait for a glitcxh that is *not* reported by the clock crate in the first floor.  
<!-- Comment: Mon Nov 26 09;11;11 comment by...s -->
<pre>We are not 100% sure we caught the glitch in the osc, because there is no time stamp.</pre>
<!-- Date: Sun Nov 25 15:31:01 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12084&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12084  width='300' align='top'></a></td><td>Store 5749: Np ~6800, Npb ~1900; Init Lumi ~152; Lostp ~13kHz, 
Lostpb ~5kHz. Decided to put Silicon in. While waiting for the Silicon bias ramping up with RC in Ready state and Aces were checking CDF clock as a part of the shot setup check list. At this moment clock error has been reported in RC. See LA traces for this glitch in the previous entry. Sent Config to Silicon crate, recovred successfuly.</td></tr></table>
<!-- Author: SashaS -->
<!-- Comment: Sun Nov 25 15;42;58 comment by...Ignacio -->
Sasha, do you mean that the Ace clock checkout caused the clock to glitch  at the beginning of the store?


<!-- Comment: Sun Nov 25 17;06;43 comment by...Ricardo -->
The clock problem was at 14:06:38 (run 254325). A pop-up window appeared in the RC console, and that's how the aces noticed. Since we were in READY I instructed the ace to reset the run. At the moment they abort the run the SRC appeared
<!-- Date: Sun Nov 25 15:46:33 2007 -->
We have Xwindow problem on b0dasvx01/02: cannot open new shell window. The error message pops up: <i>KDEInit could not launch 'konsole'</i>. If I try to start some X klient (xterm) from already running shell windows, I got the message: 
<pre> 
Xlib: connection to ":0.0" refused by server 
Xlib: No protocol specified 
 
xterm Xt error: Can't open display: :0.0 
</pre> 
We fixed this problem on b0dasvx02 by logging out. <i>svxii</i> user automatically logging in in 30sec. After that problem gone on b0dasvx02. If there is no other solution we will need to do the same (logging out/in) on b0dasvx01 (PSgui) during the quite time.
<!-- Author: SashaS -->



<!-- Date: Sun Nov 25 23:58:57 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12085&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12086  width='300' align='top'></a></td><td>VBIAS ISL B2W0L0 pink recovered</td></tr></table>
<!-- Author: marco -->


<!-- Date: Mon Nov 26 03:02:57 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12087&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12088  width='300' align='top'></a></td><td>Bias Pinkie B2W0L0, small increase, then returned to normal</td></tr></table>
<!-- Author: Joe Incandela -->


<!-- Date: Mon Nov 26 03:15:36 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12089&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12090  width='300' align='top'></a></td><td>VBIAS1 ISL B4W2L3</td></tr></table>
<!-- Author: Joe Incandela -->


<!-- Date: Mon Nov 26 03:25:27 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12097&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12098  width='300' align='top'></a></td><td>VBIAS ISL B2W0L0 and VBIAS1 ISL B4W2L2: Both now set to IGNORE</td></tr></table>
<!-- Author: Joe Incandela -->


<!-- Date: Mon Nov 26 03:26:05 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12099&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12099  width='300' align='top'></a></td><td>and here</td></tr></table>
<!-- Author: Joe Incandela -->


<!-- Date: Mon Nov 26 05:28:52 2007 -->
<pre>I put the current studies plots on b0sili01:~mmathis/SiPS/images/*.gif, since I couldn't think of a better way to display them all. There may be a couple that are blank (ISL layer 7, L00 bulkhead 0), I'll try and update them soon. </pre>
<!-- Author: Mark -->


<!-- Date: Mon Nov 26 06:41:12 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12100&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12100  width='300' align='top'></a></td><td>Store 5750, Init Lumi 158; Lostp 12kHz, Lostpb 5.5kHz. Observed shot setup from home. Since everything looked ok, gave go in with Silicon to the shift.</td></tr></table>
<!-- Author: SashaS -->


<!-- Date: Mon Nov 26 08:25:14 2007 -->
<h3>Summary of the Monitoring Shifter</h3> 
 
No problem in the checked runs. The problematic ladders that were unbiased seem OK now. Runs are getting closer to the instantaneous luminosity of the reference run for offline ADCMon plots.<br> 
I checked from run 253974 (0x3e016) to run 254128 (0x3e0b0). All looks fine.<br> 
Runs 254121, 254088 and 254094 might be useful to check possibly underbiased ladders since they have good normalization.
<!-- Author: Oscar -->


<!-- Date: Mon Nov 26 08:39:03 2007 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12102&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=silicon&action=view&page=-12102  width='300' align='top'></a></td><td>Losses during this week</td></tr></table>
<!-- Author: SashaS -->




<!-- Date: Mon Nov 26 09:20:23 2007 -->
<h2>Summary of the week</h2> 
 
<h3>Monday, November 19, 2007</h3> 
<ul> 
  <li> Setup LA on the 1st floor to spy on the clock signals <i>CDF_CLK, B0, BC, BG</i> 
  from the Tracer board installed in an empty VME create. 
  Setup external trigger from the Silicon LA to trigger on SRC error signal. 
</ul> 
 
<h3>Tuesday, November 20, 2007</h3> 
<ul> 
  <li> <b>Store 5737</b>, Init Lumi 178; Lostp ~8kHz, Lostpb ~5kHz. Decided to put Silicon in 
  <ul> 
    <li> Had clock glitch while biasing Silicon 
    <li> Fib crates turned red in RC 
    <li> Powering failed due to handshake error 
    <li> Took ~1.5 hour to recover 
  </ul> 
  <li> Tevatron separator scan. 
  Put Silicon to STBY, L00 off. Put L00 to STBY by the end of scan 
  <li> COT quadrant in SL4 is off 
  <li> Run w/o Silicon until the end of the store 
</ul> 
 
<h3>Wednesday, November 21, 2007</h3> 
<ul> 
  <li> <b>Store 5741</b>, Init Lumi 180; Lostp ~8kHz, Lostpb ~6kHz. Decided to put Silicon in. 
  <ul> 
    <li> Spontaneous switch off LB0W0, LB0W5, IB0W4, IB0W5, IB2W2 - all Fib03. 
    Ace recovered ISL ladders, L00 tripped Dvdd ovc. 
    (un-necessary) rebooting of Fib03 and power-cycling by Fib03 worked. 
    <li> Auto-Hockerization of PS6 (Fib02). 
  </ul> 
</ul> 
 
<h3>Thursday, November 22, 2007</h3> 
<ul> 
  <li> <b>Store</b> 5743, Init Lumi 175; Lostp ~16kHz, Lostpb ~5.5kHz; L1A ~8.5kHz. 
  Short run w/o Silicon: monitor losses and trigger rates. 
  Losses were smooth and going down, trigger rates ok. Decided to put Silicon in. 
  <ul> 
    <li> Auto-Hockerization of PS10 
    <li> Trigger test w/o Silicon (but Silicon was biased): heap corrupt error in Fib01 crate. 
    Rebooted crate. 
    <li> SRC Fatal Error: Resonance Detected. Restarted run. 
  </ul> 
</ul> 
 
<h3>Friday, November 23, 2007</h3> 
<ul> 
  <li>West Plug Access for COT work 
  <ul> 
    <li> Warmed up SVX (-10C -> 0C) and ISL (6C -> 15C) 
    <li> Kept monitoring DP during access 
    <li> Cooled down SVX (0C -> -6C), ISL (6C -> 10C) 
    <li> Checked-out Silicon: everything came back to the pre-access level 
    <li> Run cosmics for about 1 hour 
  </ul> 
  <li> Connected LA signal to the CDF SYNC input in U31, 
  so we now can look directly at what goes into the SRC 
</ul> 
 
<h3>Saturday, November 24, 2007</h3> 
<ul> 
  <li> <b>Store 5745</b>, Init Lumi 170; Lostp ~27kHz, Lostpb ~5.5kHz. 
  Started w/o Silicon. 
  COT tripped in SL7/8. When COT was ready, losses dropped to 10/5kHz; Lumi at this point 120; L1A ~5kHz. 
  Decided to put Silicon in. 
  <ul> 
    <li> Lost beam at 3:30 - not a quench 
  </ul> 
  <li> <b>Store 5747</b> has been lost at flattop. Abort pulled due to D0 SVX losses. 
</ul> 
 
<h3>Sunday, November 25, 2007</h3> 
<ul> 
  <li> <b>Store 5748</b>, Init Lumi 114; Lostp ~11kHz, Lostpb ~4kHz. Decided to put Silicon in 
  <ul> 
    <li> Had 10 resonances during this store. 
    <ul> 
      <Li> HRR, restarting run helped for a while then resonances happened again. 
      <li> Did not find any particular hot trigger. 
      <li> Resonances started after the UPS kicked in. 
      <li> LA traces show for all but one resonances <i>ReadBusy</i> > 30us just before SRC pulled error 
    </ul> 
    <li> TeV RF station trip. 
    <ul> 
      <li> <i>AGIGI2</i> went high; put Silicon to STBY 
      <li> Checked with MCR, that they are not going to recover tripped RF 
      <li> Put Silicon back in ASA <i>AGIGI2</i> cleared in Tevmon 
    </ul> 
  </ul> 
  <li> <b>Store 5749</b>, Init Lumi 152; Lostp ~13kHz, Lostpb ~5kHz. Decided to put Silicon in. 
  <ul> 
    <li> Clock glitch while RC in <i>Ready</i> and we were powering Silicon. 
    Config to Fib crates, recovered successfully, nothing tripped. 
  </ul> 
  <li> <b>Store 5750</b>, Init Lumi 158; Lostp 12kHz, Lostpb 5.5kHz. Decided to put Silicon in 
</ul> 
 
<h2>Conclusion</h2> 
<ul> 
  <li> Keep Silicon in healthy operation 
  <ul> 
    <li> <i>Accomplished</i> 
  </ul> 
  <li> Continue investigation of SRC/Clock errors 
  <ul> 
    <li> It seems clear that clock glitch is <i>external</i> to SRC 
  </ul> 
  <li> Observe situation with losses: 
  go in with Silicon if Lostp greater than 10kHz (current warning limit) and everything else is Ok 
  <ul> 
    <li> All but one stores came in with losses below <i>Alarm</i> limit. 
    <li> We were able to safely put Silicon in right from the beginning of the store. 
    <li> Will continue observation 
  </ul> 
</ul>
<!-- Author: SashaS -->