<!-- Title: Day shift. Tue Oct 10, 2000 -->
<!-- Leader: Alan SIll -->
<!-- Ace: Bill Orejudos -->
<!-- Co1: William Bell -->
<!-- Co2: Alexander Skiva -->
<!-- Co3: &nbsp; -->
<!-- Co4: &nbsp; -->
<!-- Co5: &nbsp; -->
<!-- Notes: -->
<pre>Supervised access in progress. 
CMP off, CMX in standby. 
Various experts taking crates.</pre>


<!-- Date: Tue Oct 10 08:58:06 2000 -->
<pre>Silicon crates have been taken by experts. 
CCAL06 has been taken by calorimeter expert. 
L1 global, L2 decision 00, 01 taken by experts. 
COT special run in progress (standalone). 
 
Rick Tesarek would like all of calorimeter for special studies soon.</pre>
<!-- Author: Alan Sill -->


<!-- Date: Tue Oct 10 09:08:53 2000 -->
Pat Lukens plans to move southwest CMX into its operating position. 
<p> 
Tried to turn CMX high voltages off in preparation for this. 
IFIX WOULD NOT RESPOND!!!  This is a problem -- one in a continuing series of IFIX / slow control difficulties (see previous log entries).  Steve Hahn went to look into it... eventually the HV's went off.  (He turned them off manually downstairs -- we need to look into the IFIX problems.)
<!-- Author: AFS AFS -->


<!-- Date: Tue Oct 10 09:23:07 2000 -->
We are using Level 1 Global, Level 2 Decision 00 and 01, SVT Fitter 06, SVT Tracking 01 and 02.  Trying to make TL2D banks with Level 1 and SVT input.  Run 100955 ended after 89 events.  Investigating L2 Processor features.
<!-- Author: Greg and Jane :: (run 100955) -->
<!-- Comment: Tue Oct 10 11;11;04 comment by...Jane -->
Returned the above crates.
<!-- Comment: Tue Oct 10 12;07;53 comment by...AFS -->
All crates except the silicon and COT have been returned.
<!-- Date: Tue Oct 10 09:44:39 2000 -->
<pre>Frank Chlebana has mods to put in, will need to reboot all crates. 
Agreed to do this just after the access if possible, about noon. 
 
Experts please be warned !!!</pre>
<!-- Author: AFS -->


<!-- Date: Tue Oct 10 10:01:44 2000 -->
<pre>CCAL06 is back.  Rick T. is taking the central for the next 10 minutes. 
Karen at Argonne would like CCAL04 and CCAL05 when this is done.</pre>
<!-- Author: Alan Sill -->


<!-- Date: Tue Oct 10 10:31:19 2000 -->
Silicon cooling tests being done by Rich Stanek and technicians. 
Silicon interlock for East side is currently tripped and disabled 
(remember to enable it when work is done).
<!-- Author: AFS -->


<!-- Date: Tue Oct 10 11:04:19 2000 -->
<pre>Took ccal06 from 9:00 to 10:00 for LED debugging purposes. 
Seems to work now.</pre>
<!-- Author: Ph.Gris -->



<!-- Date: Tue Oct 10 11:22:39 2000 -->
CMP has tripped again.  Paged expert.
<!-- Author: Alan Sill -->


<!-- Date: Tue Oct 10 11:50:31 2000 -->
<pre>CSL report: remove /logger/disk4 from the cluster for an upgrade. 
Should not affect current data taking operations. All data 
now will go to b0dau32:/logger/disk3 
</pre>
<!-- Author: TonyV -->


<!-- Date: Tue Oct 10 12:04:47 2000 -->
Access is finished.  Tried to run but TSI00 did not come back. 
Greg says to reboot it -> done.  ALso had to do cleanup on L3 farm. 
<p> 
Many people in the control room --> cleaned them out.
<!-- Author: Alan Sill -->



<!-- Date: Tue Oct 10 12:15:23 2000 -->
<pre>Work on COT Crates: 
 
 TDC's replaced in several crates. 
 XFT's replaced on several cards 
 New Tracer on b0cot06 
 "Dead" cards resurrected by power cycle. 
 
All work tested with a DAQ run, seems fine (for now)</pre>
<!-- Author: Bill Orejudos -->
<!-- Comment: Tue Oct 10 13;32;58 comment by...Ken Bloom -->
<pre>I believe Bill means "XTC," not "XFT."</pre>
<!-- Date: Tue Oct 10 12:34:25 2000 -->
<pre>Trying to take a checkout run after access. Includes calorimeter,COT, Level 1, level2 trigger crates, l3. 
 
Having problems communicating with b0tsi00.</pre>
<!-- Author: Bill Orejudos -->
<!-- Comment: Tue Oct 10 12;36;51 comment by...Bill Orejudos -->
Note: after we lost communication, logged into b0tsi00 and 
      saw 2 robin objects. We rebooted the crate and will
      try to run again using the above crates.


<!-- Date: Tue Oct 10 12:39:50 2000 -->
<pre>Next attempt (100979) undefined error in pcal04</pre>
<!-- Author: Bill Orejudos :: (run 100979) -->
<!-- Comment: Tue Oct 10 12;48;52 comment by...Bill Orejudos -->
had to reboot b0tsi00 again!!!


<!-- Date: Tue Oct 10 12:59:58 2000 -->
<pre>Another done timeout from pcal04 after 32 events</pre>
<!-- Author: Bill Orejudos :: (run 100980) -->
<!-- Comment: Tue Oct 10 13;02;44 comment by...Bill Orejudos -->
and had to reboot clock again

<!-- Comment: Tue Oct 10 13;03;09 comment by...Bill Orejudos -->
oops, of course I meant "reboot b0tsi00"


<!-- Date: Tue Oct 10 13:26:17 2000 -->
<pre>Shut down production database for upgrade.  switching to integration database</pre>
<!-- Author: rick st.denis -->


<!-- Date: Tue Oct 10 13:31:34 2000 -->
<pre>Taking a run with b0pcal00 - events seem to make it  
through the L3 processor node with no problem. Note 
that pcal00 has a new version of the front end code 
loaded.</pre>
<!-- Author: Bill Orejudos :: (run 100982) -->



<!-- Date: Tue Oct 10 13:33:05 2000 -->
<pre>stopping all run control sessions while rick st. denis 
and bill badgett do DB work.</pre>
<!-- Author: Bill Orejudos -->


<!-- Date: Tue Oct 10 14:17:23 2000 -->
<pre>The updated version of the fer has been made the default 
version. 
 
Only PCAL00 and PCAL04 have been loaded with the new software. 
 
In order to load all the crates you need to reboot them. 
 
</pre>
<!-- Author: Frank Chlebana -->


<!-- Date: Tue Oct 10 14:20:29 2000 -->
<pre>Resource Manager down for transfer to integration</pre>
<!-- Author: W.Badgett -->


<!-- Date: Tue Oct 10 14:41:42 2000 -->
Database work apparently complete -- we are advised to start running.
<!-- Author: Alan Sill -->


<!-- Date: Tue Oct 10 14:47:04 2000 -->
<pre>Jane found a telcom problem in the trigger room, so I  
seized the opportunity during the database switchover  
downtime to troubleshoot.  We discovered a subfloor cabling 
problem which we rectified.  All phone partitions in the 
trigger room are now back online.</pre>
<!-- Author: SCN -->


<!-- Date: Tue Oct 10 14:47:07 2000 -->
<pre>CMEX North East top and Storage shed west bottom 
flammable gas heads have a trouble status flag. 
Del Allspach has been contacted and he will reset them.</pre>
<!-- Author: Doug Benjamin -->


<!-- Date: Tue Oct 10 14:54:01 2000 -->
<pre>Took advantage of the database down time to study the 
behaviour of large byte count transfers to the VRB 
on most of the front-end crates. From this it is 
determined that the original kludge parameters 
need to be adjusted a bit. It is not clear whether 
or not this will fix the data corruption errors 
observed in the muon system and CLC during the 
store yesterday.</pre>
<!-- Author: patrick -->
<!-- Comment: Tue Oct 10 15;09;18 comment by...Frank Chlebana -->
The delay between 2k block transfers has been increased.

Need to reboot all crates that send data to the VRBs in order
for this change to take effect.


<!-- Date: Tue Oct 10 15:23:32 2000 -->
<pre>Noticed that CMX has tripped.  Called expert. 
 
CMP is still tripped, no response by experts to previous pages or messages. 
Paged again.</pre>
<!-- Author: Alan Sill -->
<!-- Comment: Tue Oct 10 15;34;38 comment by...AFS -->
Both Hyunsoo and Abe are now here and working on their systems.
<!-- Date: Tue Oct 10 15:27:59 2000 -->
<pre>A new way for RC to fail (just after a BUSY timeout that 
the error handler and dialogBoxPopper missed): 
Have seen 9/9 acknowledgements, (1) 
 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfefac 
SIGABRT 6 (*) abort process 
	stackpointer=0xbebfe934 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfe65c 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfe384 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfe0ac 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfddd4 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfdafc 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfd824 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfd54c 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfd274 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfcf9c 
SIGBUS 7 (*) bus error 
	stackpointer=0xbebfccc4 
 
 
Doesn't respond to CTRL-C or CTRL-Z, dies 
with kill -KILL <proc_num> 
 
</pre>
<!-- Author: SCN -->



<!-- Date: Tue Oct 10 15:45:23 2000 -->
<pre>TrigMon pager:  (630) 266-2583 
Now available!  24 hours a day, 7 days a week.</pre>
<!-- Author: Matt -->


<!-- Date: Tue Oct 10 15:46:58 2000 -->
B0 Linux machines have been quite slow, including b0dap15 used by 
the ace.  This might have something to do with b0dap30 going down about half an hour ago.  Bill has rebooted b0dap15 and Jim is looking for the problem.
<!-- Author: Alan Sill -->


<!-- Date: Tue Oct 10 15:52:43 2000 -->
<pre>b0dap30 was hung but not crashed (responded to ping). 
Reset, it came up OK.</pre>
<!-- Author: patrick -->


<!-- Date: Tue Oct 10 16:03:11 2000 -->
<pre>Resource manager is now running off of integration database. 
All Run Control users should : 
 
       setup fer devel 
       source ~cdfdaq/int 
 
until further notice.   Another log message will be posted  
when the databases revert.  All runs taken with integration  
will be lost. 
 
The production database is in the process of updgrading to  
Oracle version 6.1.6.1 
</pre><font color=red><cite>(Entry started on Tue Oct 10 16:03:07 2000)</cite></font>
<!-- Author: W.Badgett -->


<!-- Date: Tue Oct 10 16:03:13 2000 -->
<font color=red>Shift Summary: </font><pre>- Supervised access completed 
- Preparing to turn on magnet 
- Much database work this afternoon, still in progress 
- CMX, CMP HV problems (apparently now resolved) 
- various IFIX problems 
- No data taken</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   none
Delivered Luminosity   0  pb-1
Acquired Luminosity    0  pb-1
Efficiency             100%
   
</td></tr></table></pre>
<!-- Author: Alan Sill -->