<!-- Title: Owl shift. Sat Feb  8, 2003 -->
<!-- SciCo: Willis Sakumoto -->
<!-- DAQAce: G. Gomez-Caballos -->
<!-- MonAce: &nbsp; -->
<!-- CO: &nbsp; -->
<!-- OpManager: Matthew Jones -->
<!-- Notes: -->
New shift -- continuing plug sourcing

<!-- Date: Sat Feb 8 00:08:44 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158439">158439</A>
TERMINATE: Plug sourcing, PEM layers 4&6, tubes 79&80, sector 4.0
<!-- Author: oglez -->


<!-- Date: Sat Feb 8 00:15:24 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158440">158440</A>
TERMINATE: Bad run.
<!-- Author: oglez -->


<!-- Date: Sat Feb 8 00:24:58 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158441">158441</A>
TERMINATE: Plug sourcing, PHA layers 0&6, tubes 82&83, sector 4.0
<!-- Author: oglez -->


<!-- Date: Sat Feb 8 00:31:40 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158442">158442</A>
TERMINATE: Plug sourcing, PEM layers 4&6, tubes 92&93, sector 4.1
<!-- Author: oglez -->


<!-- Date: Sat Feb 8 00:37:02 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158443">158443</A>
TERMINATE: Bad run.
<!-- Author: oglez -->


<!-- Date: Sat Feb 8 01:04:44 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158444">158444</A>
TERMINATE: Plug sourcing. PHA layers 0&6, tubes 95&96, sector 4.1
<!-- Author: oglez -->


<!-- Date: Sat Feb 8 01:12:24 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158445">158445</A>
TERMINATE: Plug sourcing. PEM layers 4&6, tubes 101&102, sector 5.0
<!-- Author: oglez -->


<!-- Date: Sat Feb 8 01:12:53 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158446">158446</A>
ACTIVATE: L2 torture test
<!-- Author: Guillelmo -->


<!-- Date: Sat Feb 8 01:21:32 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158447">158447</A>
TERMINATE: Bad run. Busy timeout (partition 7 has status "recovered").
<!-- Author: oglez -->



<!-- Date: Sat Feb 8 01:28:06 2003 -->
<pre>Plug sourcing: 
Having again the problem of busy timeout. We give up the sourcing today, waiting for the problem is solved.</pre>
<!-- Author: Oscar Gonzalez-Lopez -->


<!-- Date: Sat Feb 8 01:37:38 2003 -->
Several done timeoouts from CCAL_07. I tryied recovering by <br>rebooting, but it did work. I try removing the trouble <br> card, but it also failed. I got a done timeout even with all<br> 
cards take out!, how is it possible?. Then, I stopped the run.
<!-- Author: Guillelmo :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158446' target=_top>158446</a>) -->

<!-- Date: Sat Feb 8 01:46:12 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158449">158449</A>
ACTIVATE: L2 torture again!
<!-- Author: Guillelmo -->
<!-- Comment: Sat Feb  8 01;50;00 comment by...Guillelmo -->
We are running with the standard configuration, but without<br>
CCAL_07 and CLC_00
<!-- Comment: Sat Feb  8 04;06;10 comment by...Willis -->
b0clc00 is RED/U on the System Monitor. Reboot/reset of crate
does not fix this.
<!-- Date: Sat Feb 8 01:49:14 2003 -->
We clean up L3 and Evb a while ago, because everything got <br>stuck from RC. It didn't send any message. In addition<br> 
there were a couple of problems in two EVB crates. So, we <br>decided to clean up L3 and Evb, and things were fine.<br> 
(after that, Plug sourcing guys had a problem with Evb again,<br> and decided to stop until tomorrow...)
<!-- Author: Guillelmo/Nuno -->



<!-- Date: Sat Feb 8 01:57:35 2003 -->
Muon system in stand-by (beam in the Tevatron)
<!-- Author: Guillelmo -->


<!-- Date: Sat Feb 8 01:59:51 2003 -->
Done timeout from CC_06. Shepherding worked.
<!-- Author: Guillelmo :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158449' target=_top>158449</a>) -->
<!-- Comment: Sat Feb  8 02;04;04 comment by...Guillelmo -->
It happened again. I could also recovered it.

<!-- Date: Sat Feb 8 02:08:17 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158449">158449</A>
TERMINATE: We will try to figure out what's going on in CCAl_07
<!-- Author: Guillelmo -->
<!-- Comment: Sat Feb  8 02;21;05 comment by...Guillelmo -->
Done timeout at the very beginning, but Shepherding worked.<br>
We will see if we can live for a long time...
<!-- Date: Sat Feb 8 02:17:19 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158450">158450</A>
ACTIVATE: CCAL_07 test
<!-- Author: Guillelmo -->



<!-- Date: Sat Feb 8 02:33:36 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158450">158450</A>
TERMINATE: We'll try to run cosmic...
<!-- Author: Guillelmo -->



<!-- Date: Sat Feb 8 02:50:40 2003 -->
<pre>EVB proxy craches. I had to clean up EVB. More info: 
 
Reply partition = 7, state = IDLE, result = SUCCESS at Sat Feb 08 03:31:11 EST 2003 
Partition 7: Received new RC message at Sat Feb 08 03:32:42  
State change: IDLE-&gt;Not a state  EvbProxy transition: EvbDeletePartition 
@@@ PartitionHeart(7): I refuse to send "Not a state" message at Sat Feb 08 03:32:42 EST 2003 
JuciSm::getReply(): SM timeout 100000 ms at Sat Feb 08 03:34:23 EST 2003 
Reply partition = 7, state = Not a state, result = FAILURE at Sat Feb 08 03:34:23 EST 2003</pre>
<!-- Author: Guillelmo -->

<!-- Date: Sat Feb 8 02:54:54 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158451">158451</A>
 Activated at 2003.02.08 02:52:56
<!-- Author: RunControl -->


<!-- Date: Sat Feb 8 02:54:55 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158451">158451</A>
ACTIVATE: Starting a cosmic run
<!-- Author: Guillelmo -->


<!-- Date: Sat Feb 8 03:07:59 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158451">158451</A>
 Terminated at 2003.02.08 02:55:31
<!-- Author: RunControl -->


<!-- Date: Sat Feb 8 03:08:00 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158452">158452</A>
 Activated at 2003.02.08 03:04:50
<!-- Author: RunControl -->


<!-- Date: Sat Feb 8 03:08:01 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158452">158452</A>
ACTIVATE: Cosmic run.
<!-- Author: Guillelmo -->



<!-- Date: Sat Feb 8 03:49:16 2003 -->
Done timeout from CCAL_)7. HRR helps
<!-- Author: Guillelmo :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158452' target=_top>158452</a>) -->
<!-- Comment: Sat Feb  8 04;22;32 comment by...Guillelmo -->
Another one...

<!-- Comment: Sat Feb  8 04;43;35 comment by...Guillelmo -->
Another one... 



<!-- Comment: Sat Feb  8 05;38;13 comment by...Guillelmo -->
Another one... 
<!-- Date: Sat Feb 8 04:48:05 2003 -->
Plug HV put to standby
<!-- Author: Willis -->


<!-- Date: Sat Feb 8 05:03:05 2003 -->
West CES wedge 17, strip and wire, are hot
<!-- Author: Willis :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158452' target=_top>158452</a>) -->
<!-- Comment: Sat Feb  8 05;04;10 comment by...Willis -->
That should be wedge 18...


<!-- Date: Sat Feb 8 05:42:27 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158452">158452</A>
 Terminated at 2003.02.08 05:38:59
<!-- Author: RunControl -->


<!-- Date: Sat Feb 8 05:42:28 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158452">158452</A>
TERMINATE: end cosmic run
<!-- Author: Guillelmo -->


<!-- Date: Sat Feb 8 05:56:10 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158453">158453</A>
ACTIVATE: SMX QIE calibration
<!-- Author: Guillelmo -->


<!-- Date: Sat Feb 8 05:56:11 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158453">158453</A>
TERMINATE: SMX QIE calibration
<!-- Author: Guillelmo -->
<!-- Comment: Sat Feb  8 06;00;27 comment by...Willis -->
Calibration looks okay

<!-- Date: Sat Feb 8 06:04:43 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158454">158454</A>
 Activated at 2003.02.08 06:02:52
<!-- Author: RunControl -->


<!-- Date: Sat Feb 8 06:04:44 2003 -->
Run <A HREF="/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158454">158454</A>
ACTIVATE: New cosmic run
<!-- Author: Guillelmo -->



<!-- Date: Sat Feb 8 06:08:58 2003 -->
Done time out in CCAL_07. HRR helps
<!-- Author: Guillelmo :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158454' target=_top>158454</a>) -->
<!-- Comment: Sat Feb  8 06;18;35 comment by...Guillelmo -->
Another one

<!-- Comment: Sat Feb  8 06;20;27 comment by...Guillelmo -->
Another one 

<!-- Comment: Sat Feb  8 06;24;28 comment by...Guillelmo -->
Another one


<!-- Comment: Sat Feb  8 06;31;43 comment by...Guillelmo -->
Another one 
<!-- Comment: Sat Feb  8 07;55;02 comment by...Guillelmo -->
And 13 times more...
<!-- Comment: Sat Feb  8 08;32;30 comment by...Mat -->
and another two..
<!-- Comment: Sat Feb  8 08;43;11 comment by...Frank Chlebana -->
<pre>Error (119) &gt;&gt;  (MLE) b0ccal07:Messenger:6:06:11 AM-&gt;SMXR Not Responding event 4856, slot 4, bunch ctr 68, L2buffer 0, header 0x3bfe444
Error (130) &gt;&gt;  (MLE) b0ccal07:Messenger:6:16:20 AM-&gt;SMXR Not Responding event 20645, slot 4, bunch ctr 9, L2buffer 0, header 0x3bfe409
Error (138) &gt;&gt;  (MLE) b0ccal07:Messenger:6:18:23 AM-&gt;SMXR Not Responding event 23473, slot 4, bunch ctr 18, L2buffer 0, header 0x3bfe412
Error (146) &gt;&gt;  (MLE) b0ccal07:Messenger:6:22:22 AM-&gt;SMXR Not Responding event 29285, slot 4, bunch ctr 121, L2buffer 0, header 0x3bfe479
Error (177) &gt;&gt;  (MLE) b0ccal07:Messenger:6:29:08 AM-&gt;SMXR Not Responding event 40090, slot 4, bunch ctr 3, L2buffer 0, header 0x3bfe403
Error (185) &gt;&gt;  (MLE) b0ccal07:Messenger:6:31:19 AM-&gt;SMXR Not Responding event 43547, slot 4, bunch ctr 71, L2buffer 0, header 0x3bfe447
Error (192) &gt;&gt;  (MLE) b0ccal07:Messenger:6:34:27 AM-&gt;SMXR Not Responding event 48465, slot 4, bunch ctr 27, L2buffer 0, header 0x3bfe41b
Error (3) &gt;&gt;  (MLE) b0ccal07:Messenger:6:37:23 AM-&gt;SMXR Not Responding event 52473, slot 4, bunch ctr 86, L2buffer 1, header 0x3bfe456
Error (11) &gt;&gt;  (MLE) b0ccal07:Messenger:6:39:55 AM-&gt;SMXR Not Responding event 56008, slot 4, bunch ctr 21, L2buffer 0, header 0x3bfe415
Error (18) &gt;&gt;  (MLE) b0ccal07:Messenger:6:40:36 AM-&gt;SMXR Not Responding event 56606, slot 4, bunch ctr 68, L2buffer 0, header 0x3bfe444
Error (25) &gt;&gt;  (MLE) b0ccal07:Messenger:6:41:20 AM-&gt;SMXR Not Responding event 57332, slot 4, bunch ctr 80, L2buffer 0, header 0x3bfe450
Error (38) &gt;&gt;  (MLE) b0ccal07:Messenger:6:44:00 AM-&gt;SMXR Not Responding event 60598, slot 4, bunch ctr 130, L2buffer 0, header 0x3bfe482
Error (50) &gt;&gt;  (MLE) b0ccal07:Messenger:6:46:04 AM-&gt;SMXR Not Responding event 62744, slot 4, bunch ctr 62, L2buffer 0, header 0x3bfe43e
Error (57) &gt;&gt;  (MLE) b0ccal07:Messenger:6:47:15 AM-&gt;SMXR Not Responding event 64210, slot 4, bunch ctr 24, L2buffer 0, header 0x3bfe418
Error (64) &gt;&gt;  (MLE) b0ccal07:Messenger:6:57:28 AM-&gt;SMXR Not Responding event 80210, slot 4, bunch ctr 12, L2buffer 0, header 0x3bfe40c
Error (71) &gt;&gt;  (MLE) b0ccal07:Messenger:7:14:58 AM-&gt;SMXR Not Responding event 107402, slot 4, bunch ctr 30, L2buffer 0, header 0x3bfe41e
Error (78) &gt;&gt;  (MLE) b0ccal07:Messenger:7:28:15 AM-&gt;SMXR Not Responding event 128215, slot 4, bunch ctr 18, L2buffer 0, header 0x3bfe412
Error (85) &gt;&gt;  (MLE) b0ccal07:Messenger:7:34:22 AM-&gt;SMXR Not Responding event 137701, slot 4, bunch ctr 33, L2buffer 0, header 0x3bfe421
Error (92) &gt;&gt;  (MLE) b0ccal07:Messenger:8:29:22 AM-&gt;SMXR Not Responding event 244790, slot 4, bunch ctr 0, L2buffer 0, header 0x3bfe400
Error (99) &gt;&gt;  (MLE) b0ccal07:Messenger:8:29:57 AM-&gt;SMXR Not Responding event 251037, slot 4, bunch ctr 106, L2buffer 3, header 0x3bfe46a
</pre>
<!-- Date: Sat Feb 8 06:29:12 2003 -->
<pre>HEVB error forwarded from COT_08. HRR helps. 
 
SCPU-P7-E-VrbHeader: Dump of header words for event 32606 from VRB in slot 10: 
0x00000000 0x000016a0 0x003800f8 0x00f80100 0x01380118 0x010000f8 0x01000720 0x07200100 
</pre>
<!-- Author: Guillelmo :: (run <a href='/java/cdfdb/servlet/RunSummary?RUN_NUMBER=158454' target=_top>158454</a>) -->



<!-- Date: Sat Feb 8 07:57:09 2003 -->
<TABLE BORDER=2 CELLSPACING=1 CELLPADDING=1 BGCOLOR=WHITE>
<TR BGCOLOR=YELLOW><TH>Run Number</TH>
<TH>Data Type</TH>
<TH>Physics Table</TH>
<TH>Begin Time</TH>
<TH>End Time</TH>
<TH>Live Time</TH>
<TH>L1 Accepts</TH>
<TH>L2 Accepts</TH>
<TH>L3 Accepts</TH>
<TH>Live Lumi, nb-1</TH>
<TH>GR</TH>
<TH>SC</TH>
<TH>RC</TH></TR>

<TR><TD>Totals</TD>                                                     
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD>07:55:01</TD>                                                       
<TD ALIGN=CENTER>::</TD>                                                
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD ALIGN=RIGHT></TD>                                                   
<TD> </TD>                                                              
<TD> </TD>                                                              
<TD> </TD></TR>                                                         
                                                                        
</TABLE>
<!-- Author: End of Shift Report -->



<!-- Date: Sat Feb 8 08:02:00 2003 -->
b0clc00 is not working properly. This is what we see using minicom: 
 
cd "/cdf/people1/cdf_clc/online/clcdaq"                                          
value = 0 = 0x0                                                                  
ld < receive.o                                                                   
b0dap24:bash: /cdf/people1/cdf_clc/online/clcdaq/receive.o: No such file or diry 
can't open input 'receive.o'                                                     
  errno = 0x290010 (S_netDrv_UNIX_FILE_ERROR)                                    
                                                                                 
Done executing startup script boot/front-end/startup.all.flash                   
 
Is this error normal?.  
<!-- Author: Guillelmo -->


<!-- Date: Sat Feb 8 08:04:40 2003 -->
<font color=red>Shift Summary: </font><pre> 
0000-0800 shift 
--------------- 
 1. 0000: Continuing with plug sourcing and L2 torture testing 
 2. 0000: TeV beam status: problems with Input number 5 to the 
          F2 C200 module (Dipole Correction Element PS) 
 3. 0040: first transient PSM alarm. For the last alarm seen, the 
          MUON/BMU/SVX box was red 
 4. 0105: b0clc00 "red-U": reboot or reset does not help. Removed 
          this and pcal crates from L2 torture 
 5. 0110: MCR calls and gives a 5' warning for beam into the 
          Tevatron. COT, muon, smx at standby 
 6. 0121: Alexei and Oscar finish only 2/3's of the plug 
          mini-sourcing, but give up because the BUSY TIMEOUT  
          problem has come back 
 7. 0241: solenoid powered up 
 8. 0242: L2 torture test generates very frequent DTO's from 
          ccal07. Taking out ccal07 made the DTO's go to ccal06 
          and a wcal crate. 
 9. 0255: Switching to cosmics. DTO's continue, maybe less 
          frequently: ccal07 DTO - 0347, 0421, 0442, 0536 
10. 0455: YMon - CES west wedge 18 is hot. Recalibrated Shower 
          Max 
11. 0604: New cosmic run. CES wedge 18 is not hot anymore 
          ccal07 DTO - 0617, 0618, 0622, 0629, 0631, 0634, 0641, 
          0644, 0646, 0701, 0715, 0728, 0735 (smxr slot 4 BC) 
</pre>   
<!-- Author: Willis -->