<!-- Title: 17:01:35  Thu Oct 19 2000 -->
<!-- Crewchief: CO's -->
<!-- Op1: &nbsp; -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre> use this logbook to share information about consumer operations 
 on-line...</pre>


<!-- Date: Thu Oct 19 17:08:27 2000 -->
<pre>This file is ~cdfdaq/CO_comments/CO_comments.txt 
This file is for shift comments by the COs. 
COs should feel free to write stuff which is relevant 
to CO-specific shift tasks, e.g., Event display, 
monitors, etc. 
 
Started by Rob Snihur, Elena Vataga and Konstantin Anikeev 
(under pressure from Henry Frisch)</pre>
<!-- Author: K Anikeev -->


<!-- Date: Thu Oct 19 17:09:47 2000 -->
<pre>-------------------------- 
Eve. shift, Oct. 18, 2000. 
Event display comments: 
 
1) What's the difference between cal towers and physics towers? 
2) When you pick a tower, it gives a tower address that looks like: 
   0x0xdc87670 
   etower_Node31x13 
   What are these? 
3) Why is dphi = 7.5 degrees? 
4) Is there an option to get Et instead of E in COT display? 
5) Can we get to AC++DUMP or EDMOjectLister from event display 
   (would be nice from window icon)? 
6) Trigger window shows bits in TFRD bank - is there a way to get the  
   text name from the trigger table printed out? 
   (We know this is not a trivial request.) 
7) Can we show trigger towers (from TCID bank) superposed in some way on LEGO? 
8) A side view would be nice to have.</pre>
<!-- Author: H. Frisch, Konstantin, Elena, Simona -->
<!-- Comment: Thu Nov  2 21;16;45 comment by...Matt -->
1) For most of the detector, cal trigger towers (i.e. TC1D
   towers) contain 2 physics towers in eta, and 1 in phi.  Each
   physics tower contains 2 PMTs, making 4 PMTs per trigger
   tower.  TrigMon is developing a way of reading physics tower
   energy in trigger eta-phi space for every hit in the trigger
   towers. 
6) If you look at the XMon summary file:
   /data1/consumer/results/XMon/XMonSummary_XXXXXX.dat
   where XXXXXX is the run number, you will find the Trigger
   Table for that run as well as the trigger names for each 
   FRED bit.
<!-- Date: Thu Oct 19 17:10:52 2000 -->
<pre>YMON &  TRIGMON comments: 
 
Font of Pad title are too big (Run number, time, etc.)</pre>
<!-- Author: H. Frisch, Konstantin, Elena -->


<!-- Date: Thu Oct 19 17:11:36 2000 -->
<pre>Day shift, Oct. 19, 2000. 
Event display comments: 
 
1) Having problems printing from fcdfsgi2.  From the file menu, 
   tried "Print..." which produced a file cot_1.ps of size ~6k. 
   However, the file seems to be empty. 
   Choosing "Save as canvas.ps" either rewrites the first file, 
   or again produces an empty file. 
   SOLVED: Carla had run out of disk space!</pre>
<!-- Author: Carla -->


<!-- Date: Thu Oct 19 17:12:31 2000 -->
<pre>DBANA comments: 
 
1) After making the calibration plots for say, central cal QIE, and selecting a new detector (say plug), one must click (again) on "List/List All". Otherwise, the plots will be empty!</pre>
<!-- Author: Rob -->


<!-- Date: Thu Oct 19 17:13:20 2000 -->
<pre>Misc. 
 
1) There is some really weird behaviour with the screen 
connected to b0dap24. Often, GUI windows display, but mouse 
clicks are not functional. Anyone know what's up? Observed 
with histodisplay, as well as root browser windows from 
fcdfsgi2.</pre>
<!-- Author: Rob -->


<!-- Date: Thu Oct 19 17:34:49 2000 -->
<pre>On b0dap24 /data1/consumer/histodisplay does not 
allow press any buttons (not working in plain English). 
It worked yesterday and din't work the day before ... 
Can it be WindowManager issue? 
 
Anyways, I have to run histodisplay on another machine 
(and kill the one that is being started by script 
[./startConsumers] on b0dap24).</pre>
<!-- Author: K Anikeev -->
<!-- Comment: Sat Oct 21 11;01;21 comment by...Rob Snihur -->
I'm still having this problem, and I have seen it with some other applications as well (when running from fcdfsgi2).
<!-- Date: Thu Oct 19 18:00:34 2000 -->
<pre>It would be useful to have some simple way to find Run number 
(dec -> hex and back). Also, Consumers write files with 
run number in dec  
(ex. /data1/consumer/results/YMon/YMon_101959aa.root), 
while raw data have names like 
/cdf/data08/_APHYSR.0000/ar018c0d.0005phys 
with run in hex.</pre>
<!-- Author: Elena -->
<!-- Comment: Sat Oct 21 10;40;54 comment by...Rob Snihur -->
On linux, many people use ghex.
<!-- Date: Thu Oct 19 19:00:51 2000 -->
<pre>In Event Display in LEGO options canvases does not have any information in the title on the top, so it's impossible to figure out if it is CalTower or Phys Tower or ADC Counts from the title.</pre>
<!-- Author: Elena -->


<!-- Date: Thu Oct 19 19:08:52 2000 -->
<pre>How to put 2 figures in one entry in E-log?</pre>
<!-- Author: Elena -->


<!-- Date: Thu Oct 19 23:12:48 2000 -->
<pre>It would be useful to have Bunch ID info somewhere in Event Display or Consumers.</pre>
<!-- Author: Henry Frisch, W. Ashmanskas, E.Vataga -->
<!-- Comment: Thu Nov  2 21;38;20 comment by...Matt -->
TrigMon plots the TFRD bunch counter.  It also globally checks the TFRD, TC1D, TC2D, TCMD, and TL1D bunch counters for mismatches as well as each card interally in all the banks.
<!-- Date: Thu Oct 19 23:24:24 2000 -->
<pre>I would suggest that each CO makes his/her own subdirectory 
in /cdf/people/cdfdaq on the online cluster. Then we should 
be able to have .cshrc for all of us (not to type `setup 
csdfoftb0' and `source ~cdfsoft/cdf2.cshrc' each time you 
start a new terminal window). That will also eliminate long list 
of files for cdfdaq homedirectory (as it's gonna be cleaned up), 
so each CO will only be dealing with files in his/her subdirectory (less chance of screwing up and deleting somebody 
else's files)</pre>
<!-- Author: K Anikeev -->


<!-- Date: Fri Oct 20 04:25:44 2000 -->
<pre>How can we get permission to write on /cdf/data04/s3/commission/ 
(where the validation ntuples are put)? Rob seems to have created 
his own area but I can not do this.</pre>
<!-- Author: Ashkan -->


<!-- Date: Fri Oct 20 09:03:43 2000 -->
<pre>I have set the permission to world writeable for 
fcdfsgi2:/cdf/data04/s3/commission/snihur, 
and its 3 subdirectories. 
Pasha will rename the snihur directory to something 
more official, since I cannot do this for some reason. 
</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Fri Oct 20 09:20:06 2000 -->
<pre>I copied some data validation plots made by the eve shift 
from fcdfsgi2:/cdf/home/alavi/shift/eps to  
fcdfsgi2:/cdf/data04/s3/commission/snihur/eps 
</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Fri Oct 20 15:49:41 2000 -->
<pre>DBANA displaying cal LED calibration results: 
the window that appears after you select "options" 
is different than what appeared yesterday. 
Yesterday's window matches the online instructions. 
Today's window is mostly blacked out, but I simply 
selected the bottom 3 checkboxes, and got the 
correct canvases.</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Fri Oct 20 16:47:33 2000 -->
<pre>Dmitry Litvintsev's answers to some of the questions asked 
in message #2 in this e-log: 
 
1) CalTowers are persistent objects, PhysicalTowers are 
transient (created and then deleted). PhysTowers have info 
about interaction point (currently always assumed to be 
0,0,0), thus they have info on Et, which CalTowers are lacking. 
 
2) 0x0x.... is some `technical info' (=you don't wanna know 
what it is). There's a printing mistake though (0x gets 
printed twice). What etower_... is henry already knows. 
 
3) dPhi of 7.5 degrees everywhere is a mistake. It is 7.5 only 
in plugs and 15 degrees everywhere else. Will be fixed ... 
 
4) No problem. In fact it's already implemented. Wait till 
changes go into official release. 
 
5) Yes you can. At ROOT prompt type: 
>load_lib(); 
>dump("BANKNAME"); 
>list(); <-- lists available banks 
 
6) don't know 
7) not at this time 
8) not available in the moment (and is not very high on the 
priority list) 
</pre>
<!-- Author: E Vataga, K Anikeev -->
<!-- Comment: Sat Oct 21 08;51;03 comment by...Rob Snihur -->
Re: 2) We thought PhysTowers were "cleaned up", i.e., the energy
in a central cal tower which has two PMTs is E = sqrt(E_PMT0 * E_PMT1), whereas the energy in a CalTower is E = E_PMT0 + E_PMT1.
In both cases, I believe the E_PMTi has the pedestals already
subtracted.  Single PMT spikes will therefore (not) show up in
CalTowers (PhysTowers).

BTW, I just learned how to include a "comment" in the E-Log:
click on the green bullet beside the date/time...
<!-- Date: Fri Oct 20 19:22:05 2000 -->
<pre>Dmitry Litvintsev told that "SVX/ISL 3D" canvas in mon is not available for now. 
 
Kaori suggested to remove drawing of Detector on COT canvas 
to make execution faster. (mouse right-click, choose 'toggle Display Detector' in menu)</pre>
<!-- Author: E.Vataga -->


<!-- Date: Fri Oct 20 21:51:42 2000 -->
<pre>Is it possible to include somehow AC++Dump in Consumer Display?</pre>
<!-- Author: E.Vataga -->





<!-- Date: Sat Oct 21 11:44:54 2000 -->
<pre>Stage0 monitor is spitting out many CT_Track errors of the form: 
Can't figure out z0 value. 
and 
Can't figure out cot(theta) value. 
I think this started just after the ace did a recover run. 
I have seen these errors before, but do not know th cause. 
 
I also see the usual SVT errors in TrigMon, probably 
because SVT was not included in this run. 
</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Sat Oct 21 13:47:02 2000 -->
<pre>Still getting SVT errors in TrigMon even though SVT is in the run 
(run 102109).</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Sat Oct 21 15:30:54 2000 -->
<pre>How was this CO e-log created?</pre>
<!-- Author: Rob -->


<!-- Date: Sat Oct 21 16:36:05 2000 -->
<pre>There are 4 new scripts in /cdf/people1/cdfdaq/ : 
 
 startLumMonOnly.sh 
 startStage0Only.sh 
 startTrigMonOnly.sh 
 startYMonOnly.sh 
 
As the name suggests each of them starts respective monitor. 
(Normally you run all 4, but in case one of them dies, or  
there is any other reason to (re)start just one of them these 
scripts are convinient things to use). These are basically 
stripped down versions of startConsumers.sh. You still have 
to provide partiton number. The xterm window will pop up in its 
usual place.</pre>
<!-- Author: K Anikeev -->


<!-- Date: Sat Oct 21 21:50:49 2000 -->
<pre>1) About LEGO display: 
It would be nice to make the reverse operation to picking up 
a Tower: type in coordinates and see it. Useful to check 
hot channels. 
 
2) What is the recipe to find CalTower from Event Display inside   
ConsumerDisplay plots? There are different systems: 
 
mon: 
 Tower               :0x0xe1be1b0 
 Tower               :eTower_Node33x22 
and phi/Eta coordinates 
 
Display: 
X-axis Eta Towers (in Eta units, I guess) 
Y-axis Phi Towers (number of Tower?) 
 
</pre>
<!-- Author: E.Vataga -->


<!-- Date: Sun Oct 22 06:29:40 2000 -->
<pre>b0dap26 monitors blanked out, even though the machine was alive. 
Had to reboot. Applications on the monitors on this machine as well as those of b0daq24 don't respond to mouse clicking after a 
while. Logging off and on seems to solve the problem.</pre>
<!-- Author: Ashkan -->


<!-- Date: Sun Oct 22 08:58:31 2000 -->
<pre>TrigMon aborted during the zerobias run 102150 
we were taking when our shift started. 
I verified there was >2GBytes free in /data1. 
The problem seems to be "storage exhausted" - 
but WHAT storage? 
I grabbed a whole lot of output which follows. 
 
</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Sun Oct 22 09:01:52 2000 -->
<pre>Here's the TrigMon output I promised... 
 
StorableBank::get_I4_element(): out-of-bounds exception 
offset = 0, index = 15, data_size = 15 
out-of-bounds exception occurred while accessing Bank SVTD, 1 
Fatal in <operator new>: storage exhausted 
aborting 
Abnormal termination caused by signal: 6 
 CalTrigger aborting !  
 **********************************  
 RUN NUMBER =  102150 
 TRGSim SUMMARY AFTER 1201 EVENTS  
 **********************************  
 ********** DIRAC SUMMARY ********** 
 ********** DIRAC BOUNCH COUNTER SUMMARY ********** 
 ********** CRATESUM SUMMARY ********** 
 ********** CRATESUM BOUNCH COUNTER SUMMARY ********** 
 ********** CRATESUM/DIRAC BC SUMMARY ********** 
 ********** HOT CABLES SUMMARY  *********** 
 ********** DCAS  SUMMARY  *********** 
Display Server: Received string message: END CONNECTION 
                ==> Terminating connection to the Consumer at 
                b0dap19.fnal.gov 
SysError in <TFile::WriteBuffer>: error writing to file server_at9052.root (Stale NFS file handle) 
SysError in <TFile::WriteBuffer>: error writing to file server_at9052.root (Stale NFS file handle) 
SysError in <TFile::WriteBuffer>: error writing to file server_at9052.root (Stale NFS file handle) 
SysError in <TFile::WriteBuffer>: error writing to file server_at9052.root (Stale NFS file handle) 
Display Server: There is no active socket from the consumer. 
                ==> going to RETURN NOW ! 
TrigMonModule: Root file /data1/consumer/results/TrigMon/TrigMon_102150aa.root saved. 
 
 *** Break *** segmentation violation 
</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Sun Oct 22 09:10:24 2000 -->
<pre>LumMon aborted, apparently due to a problem with the 
socket connection to the CSL. 
Output follows. 
 
Error in <TFile::TFile>: file /data1/consumer/results/LumMon/LumMon_102150aa.root already exists 
LumMonModule: Root file /data1/consumer/results/LumMon/LumMon_102150ab.root saved. 
 
 *** Break *** segmentation violation 
Abnormal termination caused by signal: 6 
Display Server: Received string message: END CONNECTION 
                ==> Terminating connection to the Consumer at 
                b0dap24.fnal.gov 
Display Server: There is no active socket from the consumer. 
                ==> going to RETURN NOW ! 
LumMonModule: Root file /data1/consumer/results/LumMon/LumMon_102151aa.root saved. 
</pre>
<!-- Author: Rob Snihur -->
<!-- Comment: Mon Oct 23 15;45;13 comment by...Napora -->
<pre><BR>
<BR>This error has continued to occur -- apparently LumMon has trouble 
<BR>terminating correctly at the end of a run.  Probably not critical, 
<BR>but it should be looked into.</pre>
<!-- Date: Sun Oct 22 09:13:32 2000 -->
<pre>Once again, histodisplay does not respond to mouse clicks on 
b0dap24.  I am using GNOME.  Will try it on another machine. 
</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Sun Oct 22 09:43:40 2000 -->
<pre>Another LumMon abort, again looks like a socket connection problem.</pre>
<!-- Author: Rob Snihur -->


<!-- Date: Sun Oct 22 09:46:22 2000 -->
<pre>Another TrigMon abort - same problem as above.</pre>
<!-- Author: Rob -->


<!-- Date: Sun Oct 22 11:26:13 2000 -->
<pre>Tried running event display on node b0dap17, 
but sending windows to b0dap24.  Does not 
respond to mouse clicks.  Still running GNOME on b0dap24. 
Just trying to gather info on this problem...</pre>
<!-- Author: Rob -->


<!-- Date: Sun Oct 22 11:43:11 2000 -->
<pre>TrigMon died again as above.</pre>
<!-- Author: Rob -->


<!-- Date: Sun Oct 22 12:32:09 2000 -->
<pre>LumMon died again - same way as above.</pre>
<!-- Author: Rob -->


<!-- Date: Sun Oct 22 13:04:41 2000 -->
<pre>trigmon died again as above.</pre>
<!-- Author: rob -->


<!-- Date: Sun Oct 22 14:12:50 2000 -->
<pre>Jane says the TrigMon problem may be a memeory leak 
problem, but is difficult to be certain, as "purify" 
only runs on fcdfsgi2.  We tried rebooting b0dap19, 
and we will see if it dies again. 
 
Also, LumMon died again as above. 
</pre>
<!-- Author: Rob -->


<!-- Date: Sun Oct 22 14:48:47 2000 -->
<pre>LumMon died.</pre>
<!-- Author: rob -->


<!-- Date: Mon Oct 23 09:26:38 2000 -->
<pre>TrigMon crashed -- we re-ran it and are getting some new errors: 
         
%ERLOG-e SL1D: Sync error 5.13245e+09 detected 23-Oct-2000 09:23:51  
         run = 102216 event = 42478 
%ERLOG-e Global Bunch Counter: Mismatch between TFRD and TCMD: 0 8  
         23-Oct-2000 09:23:52 run = 102216 event = 42478 
%ERLOG-e No real XTRD_StorableBank found- track input is 0xff!:  
         /cdf/code-Linux-2.0.34/cdfoffline/dist/cdfsoftb0/3.9.0f/CalTrigger/src/CalTriggerEvent.cc  
        : 532   CalTriggerDataMaker::event() 23-Oct-2000 09:23:52  
         run = 102216 event = 42560 
%ERLOG-e SL1D: Sync error 5.13852e+09 detected 23-Oct-2000 09:23:52  
         run = 102216 event = 42560 
</pre>
<!-- Author: Rob -->
<!-- Comment: Thu Nov  2 21;43;45 comment by...Matt -->
These errors are the kind of thing that TrigMon is designed to monitor and report.  For example, if the COs are seeing Global Bunch Counter: Mismatches in every event (this can be checked in the Global Bunch Counter Check plot) this should be reported to the ace.
<!-- Date: Tue Oct 24 03:16:29 2000 -->
<pre>TrigMon has gone out...  It does not work any more..</pre>
<!-- Author: S. Woo -->
<!-- Comment: Thu Nov  2 21;45;57 comment by...Matt -->
Wherever TrigMon went out to, it seems to have come back.
<!-- Date: Tue Oct 24 09:49:47 2000 -->
<pre>We ran out of disk space this morning causing problems with 
everything the CO touches (monitors, event display, Linux ...).  
This is probably what caused TrigMon's problem in the previous 
entry, but there is no way to be sure.</pre>
<!-- Author: Napora -->
<!-- Comment: Tue Oct 24 09;55;50 comment by...Napora -->
<pre><BR>
Oh yeah, the problem <I>has</I> been reported and is currently being worked on by the
experts.  Run control is down for the moment too.</pre>


<!-- Date: Tue Oct 24 11:30:37 2000 -->
<pre>LumMon continues to quit.  We continue to keep restarting with  
startLumMonOnly.sh</pre>
<!-- Author: Olga -->


<!-- Date: Tue Oct 24 12:39:09 2000 -->
<pre>The problem of running out of disk space has been solved and a 
run was taken, however now we are enountering a problem with the 
oracle database and running has been halted again...  </pre>
<!-- Author: Napora -->


<!-- Date: Tue Oct 24 13:04:57 2000 -->
<pre>Pasha Murat came by to show us how to run the Production Exe.  
After a <B><I>very</I></B> confusing explanation the code 
crashed when we tryed to run it anyways.  The appropriate 
experts (i.e. guilty parties) parties -- Rick Snider, Liz 
Sexton, and Rob Kennedy -- have been informed.</pre>
<!-- Author: Olga and Napora -->
<!-- Comment: Tue Oct 24 13;05;51 comment by...Rob -->
<pre>Pash called and asked us to try to run it again using the frozen
release 3.9.3 -- we will try it when we get a chance.</pre>


<!-- Date: Tue Oct 24 14:01:15 2000 -->
<pre>TrigMon crapped out again.  Here is a sample of the errors: 
 
 After 246 events, eta = 23, phi = 17 HAD tower energy disagrees 42 times 
 After 246 events, eta = 23, phi = 18 EM tower energy disagrees 36 times 
 After 246 events, eta = 23, phi = 18 HAD tower energy disagrees 42 times 
 After 246 events, eta = 23, phi = 23 EM tower energy disagrees 37 times 
 After 246 events, eta = 23, phi = 23 HAD tower energy disagrees 48 times 
TrigMonModule: Root file /data1/consumer/results/TrigMon/TrigMon_102285aa.root saved. 
Display Server: Client connected from b0dap14.fnal.gov 
 
This is only a subset, it actually occured for pretty much all of the towers.</pre>
<!-- Author: Rob -->
<!-- Comment: Fri Oct 27 01;10;57 comment by...Jane Nachtman -->
This is not an example of TrigMon "crapping out", but rather TrigMon doing its job. 

The first messages indicate that CalTriggerMonitor has found differences between the Calorimeter trigger simulation and data banks.  When you see this error, you should consult the Ace and SciCo to see if there is anything unusual going on with those crates, for example if some of the Admem or Dirac or Dcas crates have been removed from the run.  A calorimeter expert may need to be consulted to try to understand the errors.
  
The message "TrigMonModule: Root file ..." just tells you where this run's root file has been saved to.   Again, this shows up at the end of every run.
<!-- Date: Tue Oct 24 15:04:35 2000 -->
<pre>The procedure to run Production executable is as follows: 
 
First log into your account on fcdfsgi2 (if you are an online 
machine in the control room, you have to first ssh to cdfsga 
and from there ssh to fcdfsgi2) 
 
Then do the following: 
 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoft2 development 
newrel -t development test 
cd test 
addpkg -h Production 
 
cp /cdf/data04/s3/commission/doc/setup_input.tcl Production/. 
cp /cdf/data04/s3/commission/doc/setup_output.tcl Production/. 
 
ProductionExe Production/run_Production.tcl >& Production.output 
 
After copying (doing the cp commands above) the .tcl files, you 
need to look in setup_input.tcl and change the input file to the 
particular run you want to run production on. 
 
 
As Rob wrote earlier we tried this with development and it  
crashed.  Liz, Rob, and Rick were notified.</pre>
<!-- Author: Olga -->


<!-- Date: Tue Oct 24 15:14:09 2000 -->
<pre>In the little time we had to look at plots, we noticed the following problem channels in the calorimeter. 
 
Run 102272 (we looked at this around 10:30am this morning) 
 
CEM  
West	Left	eta=2	phi=1	DEAD 
East	Right	eta=7	phi=21	HOT (0.036%) 
 
CHA 
West	Left	eta=-5 	phi=10	HOT 
West	Left	eta=-7	phi=18	DEAD 
West	Left	eta=-7	phi=7	DEAD 
East	Left	eta=7	phi=21	HOT 
East	Right	eta=0	phi=6	HOT 
 
</pre>
<!-- Author: Olga & Rob -->


<!-- Date: Tue Oct 24 15:56:54 2000 -->
There is now a second version of YMon that needs to be run in 
addition to the regular four monitors.  This is so that we can 
get some additional COT plots during the shot.  To run this 
second YMon do the following: 
 
     &gt; setup cdfsoft2 special_3_level3_3.9.0 
 
     &gt; runYMon &lt;id&gt; 
 
Where &lt;id&gt; is the partition number.  The point of contact 
concerning problems/questions with this is Greg Veramendi (x8617, 
mendi@fnal.gov)
<!-- Author: Napora -->


<!-- Date: Tue Oct 24 15:59:36 2000 -->
<pre>Had some HTML formatting problems with that last entry, hope  
this looks better (it is the exact same info). 
================================================================ 
There is now a second version of YMon that needs to be run in 
addition to the regular four monitors. This is so that we can get 
some additional COT plots during the shot. To run this second 
YMon do the following:  
   
     setup cdfsoft2 special_3_level3_3.9.0 
     runYMon id 
 
Where id is the partition number. The point of contact 
concerning problems/questions with this is Greg Veramendi (x8617, 
mendi@fnal.gov)</pre>
<!-- Author: Napora -->


<!-- Date: Tue Oct 24 16:10:17 2000 -->
<pre>LumMon is still behaving like a freak.  It still does not end  
gracefully when a run ends (same error as before).  To add insult 
to injury this crash frequently results in the system thinking  
that LumMon is still running even though it is not.  The upshot 
of this is that when you try to restart LumMon it gives you an error: 
 
<B>LumMon is already running on this machine</B> 
 
To fix this open a new xterm and run the command 
 
<B>jobs</B> 
 
This will list all of the current jobs from your account.  Find 
the number of the LumMon job and run the command 
 
<B>kill -9 %<I>jobnum</I></B> 
 
Where <I>jobnum</I> is the job number you just looked up.  This 
will kill the fake LumMon in memory and then you can restart.</pre>
<!-- Author: Napora -->


<!-- Date: Wed Oct 25 23:23:04 2000 -->
<pre>New executable for LumMon,YMon,TrigMon,Stage0,and XMon Built. 
Here is how to run each. 
 
ssh b0dap24 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoft2 development 
cd test310rel 
runLumMon 0 
 
ssh b0dap14 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoft2 development 
cd test310rel 
runYMon 0 
 
ssh b0dap26 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoft2 development 
cd test310rel 
runStage0 0 
 
ssh b0dap19 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoft2 development 
cd test310rel 
runTrigMon 0 
 
ssh b0dap19 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoft2 development 
cd test310rel 
runXMon 0 
 
</pre>
<!-- Author: Koji and Justin -->


<!-- Date: Wed Oct 25 23:53:27 2000 -->
<pre>Tested new version of LumMon,YMon,TrigMon,Stage0, and Xmon worked.  Went back to old version after testing.</pre>
<!-- Author: Koji and Justin -->


<!-- Date: Thu Oct 26 23:06:51 2000 -->
<pre>To Run Ymon,TrigMon,Stage0,LumMon,and XMon: 
 
LumMon 
ssh b0dap24 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoftb0 3.10.0a 
runLumMon 0 
 
YMon 
ssh b0dap14 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoftb0 3.10.0a 
runYMon 0 
 
Stage0 
ssh b0dap26 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoftb0 3.10.0a 
runStage0 0 
 
TrigMon 
ssh b0dap19 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoftb0 3.10.0a 
runTrigMon 0 
 
XMon 
ssh b0dap19 
source ~cdfsoft/cdf2.cshrc 
setup cdfsoftb0 3.10.0a 
runXMon 0 
 
Event Display 
setup cdfsoft2 -g kor 
Mon 0 
 
</pre>
<!-- Author: Justin and Koji -->
<!-- Comment: Fri Oct 27 15;39;02 comment by...napora -->
For those of you who like to cut and paste from here, the last entry should be

mon 0

(Lower case M.)
<!-- Date: Thu Oct 26 23:08:34 2000 -->
<pre>Lego plots must be printed the following way to save toner: 
 
evdprint file.ps | flpr -q b0ctl_tcolor</pre>
<!-- Author: Koji and Justin -->
<!-- Comment: Fri Oct 27 13;49;11 comment by...Napora -->
<pre><BR>When I do this I get an error saying

<BR><B>evdprint: Command not found.</B> 

<BR>How do we set this up to work properly?
</pre>
<!-- Date: Fri Oct 27 07:08:29 2000 -->
<pre>There is text output file of XMon for each run. Consumers may have to keep printing those.. 
The file name is XMonSummary_<runNumber>.dat in b0dap19.</pre>
<!-- Author: S.Woo -->
<!-- Comment: Thu Nov  2 21;48;49 comment by...Matt -->
XMon output is now in the /data1/consumer/results/XMon area.  Every 5 runs or so (or at the request of the shift crewers) COs should print out the .dat files for each run and put them in the big orange XMon binder.  Thanks!
<!-- Date: Fri Oct 27 13:17:09 2000 -->
<pre>Pasha came by and showed us a nice way to skip events in the  
offline event display.  From the root prompt type: 
 
<B>Ac->DisablePath("AllPath"); Ac->Continue(100); Ac->EnablePath("AllPath"); Ac->Continue(1);</B> 
 
Where 100 is the number of events that you want to skip.  You can  
enter this all on one line or as separate lines.   
 
The commands are also in a macro file called evd/jump.C for the 
cdfdaq account.  Unfortunately, you will have to port it over to 
fcdfsgi2 yourself to use it with your account there.  Pasha plans 
to include this macro file in the next release, but it will 
probably take a few days. 
 
To run the macro from root type 
 
<B>.L (full path)/jump.C;</B> 
 
Now you can jump events by typing 
 
<B>jump(n);</B> 
 
Where n is the number of events that you wish to skip.   
</pre>
<!-- Author: Napora -->
<!-- Comment: Fri Oct 27 16;07;12 comment by...Napora -->
That jump.C macro is on the b0dap17 machine
<!-- Date: Fri Oct 27 15:40:10 2000 -->
<p>Request from Kaori:  when using a Level 3 executable with jet 
clustering, watch for error logger messages from 
<tt>CdfMetPuffer</tt>.  Please clip a few of these messages 
and e-mail them to <a href="mailto:maeshima@fnal.gov">Kaori</a>. 
This message comes from the consumer input module.</p>  
<!-- Author: Jeff -->


<!-- Date: Fri Oct 27 15:44:46 2000 -->
<p><tt>XMon</tt> needs to be restarted at the beginning of 
each run for the time being.  Otherwise it'll pick up the 
wrong trigger table when a new run is started.</p>  
<!-- Author: Jeff -->
<!-- Comment: Thu Nov  2 21;52;05 comment by...Matt -->
This should now be fixed - XMon will pick up the correct trigger table from the DB without needing a restart.  We are testing this theory tonight.
<!-- Date: Sat Oct 28 00:33:20 2000 -->
<pre>For the five monitoring windows, now the setup is again 
 
         setup cdfsoftb0. 
 
And with ONLY one command './startConsumers.sh 0', you can start all the five windows at once just as before.  
 
If one of them die, use the same command once more after killing the dead window.  
 
We don't need different commands for each monitoring windows any more!!!!!</pre>
<!-- Author: S.Woo -->


<!-- Date: Sat Oct 28 06:10:57 2000 -->
<pre>There a hot tower in Central Hadronic. For the following tower the Left PMT occupancy per event is 1. 
 
    CHAD East    Wedge 21, Eta 7 
 
Run number: 102645 and more  (Always through the night) 
</pre>
<!-- Author: S.Woo -->


<!-- Date: Sat Oct 28 06:15:45 2000 -->
<pre>The upper hot tower makes the COT display in eventdisplay funny. 
That is if there's nothing the COT display always shows 0.17GeV in hadronic calorimeter.</pre>
<!-- Author: S.Woo -->


<!-- Date: Sat Oct 28 14:03:47 2000 -->
<pre>We keep getting an error where we have a consumer monitor 
running, but it doesn't show up as having an open socket.  The 
problem that just prompted me to make this entry was with both 
YMon and TrigMon.  I restarted them and noticed in their starting 
messages that they claimed to connect to ports 9054 and 9052 
respectively.  They still do not show up on the connection status 
pages.  I do seem to be able to connect to them on those socekts 
but when I then click on start the consumer display gui locks up. 
 
I will add additional comments here as other problems like this 
occur.</pre>
<!-- Author: Napora -->
<!-- Comment: Sat Oct 28 14;33;26 comment by...Napora -->
<pre>Disregard the port numbers I typed above.  I think they may
have been the consumers socket to the run, <em>not</em> the one for 
histodisplay to log into. </pre>


<!-- Date: Sun Oct 29 04:53:35 2000 -->
<pre>Consumers are supposed to look at the following 2 runs with event display. Print the COT output for the events which have the trigger combination of 4&21 or 21&15 or 20&22. 
 
The saved root files are in /cdf/data08/_APHYSR.0000/ on fcdfsgi2. But.. there was some problem I met. While I was runing evd on fcdfsgi2, it keeps on being corrupted. I couldn't find why.</pre>
<!-- Author: S.Woo -->


<!-- Date: Sun Oct 29 05:06:27 2000 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-2&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-2  width='250' align='top'></td><td>COT plot for the events which has trigger combination of 4&21 or 21&15 or 20&22 for run number 102592, 102593.</a></td></tr></table>
<!-- Author: S.Woo -->


<!-- Date: Sun Oct 29 05:07:42 2000 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-3&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-3  width='250' align='top'></td><td>One more..</a></td></tr></table>
<!-- Author: S.Woo -->


<!-- Date: Sun Oct 29 16:02:49 2000 -->
<pre>I have added the  <a href="/ops/COGuide/index.html">The New, High Speed, Low Drag Consumer Operator's 
Guide</a> to the CO Help page.  It still needs a <em>little</em> work, but it is 
almost there!</pre>
<!-- Author: Napora -->


<!-- Date: Sun Oct 29 19:32:23 2000 -->
<pre>Created Muon Plots for run 102580 (X190B4). 
Greg created a new executable for the new code. 
Copied and edited the .tcl located in  
/cdf/scratch/mendi/testrel/YMon/.   The executable was  
run from /cdf/scratch/mendi/bin/IRIX6-KCC_3_3/YMon/ with  
the following command "YMon /path/*.tcl". 
 
</pre>
<!-- Author: Justin and Yoshi -->
<!-- Comment: Mon Oct 30 09;39;00 comment by...Greg V -->
<pre>This is only a temporary executable made with the code
that I was developing last night. It should soon be
in development. This will no longer work.</pre>
<!-- Date: Sun Oct 29 19:49:05 2000 -->
<pre>To run the histodisplay, run Histo(tab) from the same  
directory specified in the previous log entry.</pre>
<!-- Author: Koshi and Justin -->


<!-- Date: Mon Oct 30 00:28:35 2000 -->
<pre>Koichi and Niccolo is on shift. </pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Mon Oct 30 07:54:37 2000 -->
<pre>I copied this file into the svxii account home directory area.  
 
> cp /usr/doc/netscape-common-4.75/Netscape.ad ~svxii/Netscape  
 
Then, I changed the parameter for 'saveHistoryInterval' from  
600sec to 86400sec (24hour).   
 
*saveHistoryInterval:           86400  
 
I run the netscape during my owl shift.  
I've never seen that message.   
But, you should see the one after running 24 hour, unless   
you change the parameter.   
 
Let me know if you have a trouble.   
 
P.S. if you're using the netscape other than 4.75,   
it will complain. In that case, you have to rename   
the file name.   
</pre>
<!-- Author: Koichi -->
<!-- Comment: Mon Oct 30 08;09;06 comment by...Koichi -->
<pre>This does not work on b0dap17. <BR>
Use another machine. or Use xrdb like this.<BR>
<BR>
> xrdb -m ~/Netscape.4.73<BR>
<BR></pre>



<!-- Comment: Tue Oct 31 06;10;12 comment by...Koichi. -->
Fixed the problem on b0dap17.
You should see the message only 1/day.
<!-- Date: Mon Oct 30 10:37:55 2000 -->
<pre>At 10:35Am,b0dap24 crashed, restart it.</pre>
<!-- Author: kai -->


<!-- Date: Mon Oct 30 11:30:53 2000 -->
<pre>Run 102790 and 102792 
On the Calorimeter 1d plots of Wedge Occupancy,  
 
phi=12 PEM is dead (empty) 
phi=34 PHA is dead (empty) 
 
</pre>
<!-- Author: Olga -->


<!-- Date: Mon Oct 30 11:47:33 2000 -->
<pre>LUMON: run=102790 clc time channel 16 is dead.</pre>
<!-- Author: Olga &Kai -->


<!-- Date: Mon Oct 30 11:49:54 2000 -->
<pre>Calorimeter 
Run 102790 
WHA phi=41-47 are dead (from 1D plots of Wedge occupancy) 
Is there a power supply gone here? 
Please inform CO's of situation.</pre>
<!-- Author: Olga & Kai -->


<!-- Date: Mon Oct 30 15:23:19 2000 -->
<pre> 
Run 102800 
Stage0 is giving error message: 
CT_Track: Can't figure out z0 value 
CT_Track: Can't figure out cot(theta) value 
 
These error messages are repeating themselves  
over and over, and are the only messages we can receive from  
Stage0.  Seems pathological and should be fixed. 
</pre>
<!-- Author: Olga & Kai -->


<!-- Date: Mon Oct 30 15:40:54 2000 -->
<pre>The CT_Track error message from Stage0 is NOT 
pathological.  Talked to Kevin Burkett and he said  
this message comes from tracking code when there is 
an axial track.  It shouldn't be an alarm. 
</pre>
<!-- Author: Olga -->


<!-- Date: Mon Oct 30 17:39:10 2000 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Mon_Oct_30_2000_173903.ps align=top><img src=/elog/gifs/ghost64.gif  width='56' align='top'></td><td>There is disagreement between data and trigger emulation DCAS Em Towers 23 and 24 (eta); phi 13 The same towers are in disagreement also in Hadronic calorimeter</a></td></tr></table>
<!-- Author: Stefano, Saverio -->
<!-- Comment: Mon Oct 30 17;40;17 comment by...Stefano & Saverio -->
<pre>Run 102807</pre>

<!-- Comment: Mon Oct 30 18;03;52 comment by...Sav -->
The same discrepancy was present in the 3 previous runs. 


<!-- Date: Mon Oct 30 18:34:08 2000 -->
<pre>CLT asks to keep an eye on some particular hists: 
Lumon--> CLTD --> CLTD_Occupancy_canvas 
All 6 hists should be filled. If not, call CLT expert.</pre>
<!-- Author: Saverio and Stefano -->


<!-- Date: Mon Oct 30 18:35:06 2000 -->
XMon crashes often
<!-- Author: {Quick_Entry} -->
<!-- Comment: Mon Oct 30 22;18;40 comment by...Matt -->
<pre>This is being investigated.</pre>
<!-- Date: Mon Oct 30 19:15:02 2000 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-5&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-5  width='250' align='top'></td><td>Run 103818 Trigger towers are  still hot and in disagreement with emulation.</a></td></tr></table>
<!-- Author: Saverio, Stefano -->
<!-- Comment: Mon Oct 30 21;17;00 comment by...R.J. Tesarek -->
<br><b>Problems Observed in TrigSim:</b>
<br>The trigger towers (eta=22,23 phi=13) correspond to L1 crate
b0l1cl03, DIRAC slot=9.  Checked the cables between DIRAC and DCAS and find everything plugged in correctly.  This doesn't look
like a cable problem upstream of DIRAC since the DIRAC occupancy
in TrigSim does not show any peaks in the same region.
<br>
<!-- Date: Mon Oct 30 19:40:30 2000 -->
<pre>Carla and Greg have been informed of this problem. They were 
aware of it but it did not seem to cause trigger rate problems. 
They are going to investigate this problem, it may be due to 
a bad connection of some sort.</pre>
<!-- Author: Stefano and Saverio -->


<!-- Date: Mon Oct 30 20:47:13 2000 -->
<pre>Run 102823. Bodap24 freezes. Close all consumer monitors, 
 log-off, start again.</pre>
<!-- Author: Stefano, Saverio -->




<!-- Date: Mon Oct 30 22:43:22 2000 -->
<pre>In HistoDisplay --> connection --> status the number of events 
is not updated. </pre>
<!-- Author: Sav -->


<!-- Date: Mon Oct 30 23:29:47 2000 -->
<pre>We have all monitors running but we can see only  
LuMon and XMon from the consumer display. 
restarting the consumer display did not help. 
We try starting again the monitor consumers.  
...not nice during a run</pre>
<!-- Author: Saverio & Stefano -->


<!-- Date: Tue Oct 31 00:49:49 2000 -->
<pre>Koichi and Niccolo is on shift. </pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Tue Oct 31 01:09:21 2000 -->
<pre>X-window is down on b0dap19 after logging out. </pre>
<!-- Author: Koichi, Niccolo -->
<!-- Comment: Tue Oct 31 05;51;24 comment by...Koichi , Niccolo -->
<pre>b0dap17 too.<BR>
You have to start X after you log in. <BR>
> startx & <BR>

</pre>



<!-- Date: Tue Oct 31 08:55:56 2000 -->
<pre>run 106862 keeps giving error message like this: 
Offlinr Software Location does not match location given by DB: 
channel 122 module ID  507 SL: 7 SL_DB: 7 cell: 3 cell_db:483 
wire 9 width 25 star 186 crate 10 slot 9: chip 94 
 
.....  
 
</pre>
<!-- Author: kai -->


<!-- Date: Tue Oct 31 10:28:10 2000 -->
<pre>Greg Veramendi said it might be useful  
for CO's to print out COTDOccupancy1dCrateXX  
plots for Crates 0-19 (XX=crate number).   
These are in the COTD file in YMON.   
You can divide the ROOT canvas into 20 pads, 
(right click on canvas to get the divide option)  
and then plot each crate in a different pad.   
(middle click on pad you want plot to appear in) 
 
This will help see if there is anything wrong with 
COT crates.  Not every slot has a TDC, so some slots  
should be empty, but if there is a slot that is not 
empty but has a low occupancy compared to nearby slots, 
 this may be a problem and COT expert should be notified.  
 
Some of these plots are in the YMON notebook, if you want  
to see for reference.</pre>
<!-- Author: Olga -->


<!-- Date: Tue Oct 31 10:32:50 2000 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Tue_Oct_31_2000_103214.ps align=top><img src=/elog/gifs/ghost64.gif  width='56' align='top'></td><td>Run 102264 20 COT crate 1D occupancy plots</a></td></tr></table>
<!-- Author: Olga -->


<!-- Date: Tue Oct 31 10:38:43 2000 -->
<pre>Nothing seems to have changed in the  
calorimeter.  The two wedges in the  
Plug that were dead yesterday are still 
dead today, and the wedges thate were dead in the  
Wall yesterday are still dead today. 
 
COT looks ok, there are a few slots which seem 
to have a low occupancy.  See the graph in the  
previous entry. 
 
</pre>
<!-- Author: Olga -->


<!-- Date: Tue Oct 31 11:19:00 2000 -->
<pre>Talked to Avi about COT plots.  He said the 1D plots 
of occupancy vs slot number for every crate (20 crates) 
are not so helpful to diagnose big problems in the COT.   
We should instead look at the plots: 
COTDStartSL1, COTDStartSL2, ..., COTDStartSL7 
 
The start time peak should begin at about 140 ns (the x-axis is in units of nanoseconds and the average t_0 is 140 nanoseconds), and should be about 200 ns wide.  If there are problems, there will be a big peak before the one starting at 140ns.   
 
**YOU CAN ONLY LOOK AT THIS IF THE RUN WAS NOT TAKEN IN MYRON 
MODE***  If it was taken in Myron mode, there will be more than  
one start time peak.    
 
You can compare these 1D start time plots with the plots, 
COTDMultiplicity2dSL0Canvas, COTDMultiplicity2dSL1Canvas, 
...,COTDMultiplicity2dSL7Canvas.  There can only be a total 
of 8 hits in these plots.  There should only be about an  
average of 2-4 hits.  If there is a cell with 8 hits, it is hot, 
and these are probably the cells/wires that are causing the 
noise in the 1 D start time plots. 
 
 
</pre>
<!-- Author: Olga -->


<!-- Date: Tue Oct 31 12:21:57 2000 -->
<pre>Run 102872 
After taking a closer look at the 2d pmt occupancy 
plots for the calorimeters, we find many hot channels,etc. 
 
We're not sure if some of them are new hot channels.  Those 
that seem to have drastically changed since yesterday evening 
are: 
 
CHA  
East: eta=6, phi=4  HOT 
 
CEM  
East: eta=1, phi=20,21 HOT 
East: eta=1, phi=4  HOT 
East: eta=2, phi=0 HOT 
</pre>
<!-- Author: Kai & Olga -->


<!-- Date: Tue Oct 31 16:50:58 2000 -->
<pre>YMon crashed due to segmentation violation  
(abnormal termination due to signa: 6). 
Program restared. 
(run 102888)</pre>
<!-- Author: Saverio & Stefano -->


<!-- Date: Tue Oct 31 16:54:16 2000 -->
<pre>Many XMon error messages: 
SysError in <TFile::TFile>: file /data1/consumer/results/XMon/Xmon_102888xx.root  
can not to be opened (Permision denied). 
(run 102888)</pre>
<!-- Author: Saverio & Stefano -->
<!-- Comment: Tue Oct 31 17;39;47 comment by...J & M -->
Fixed.


<!-- Date: Tue Oct 31 21:54:08 2000 -->
<pre>Trigmon crashed due to segmentatin violation.</pre>
<!-- Author: Saverio + Stefano -->


<!-- Date: Tue Oct 31 22:05:24 2000 -->
<pre>TrigMon crash everytime. We saved the log in  
/cdf/people1/cdfdaq/TrigMon_crash.out. 
(run 102906)</pre>
<!-- Author: Stefano & Saverio -->
<!-- Comment: Tue Oct 31 23;45;47 comment by...Stefano & Saverio -->
This is due to the lack of calorimeter crates in the DAQ
For this kind of runs only CLC was included.

<!-- Comment: Tue Oct 31 23;47;41 comment by...Jane -->
Thanks for saving the log file!  These crashes are due to the fact that special runs were being taken that did not include the calorimeter crates.  When there are no TC1D and TC2D banks present (from the Level 1 and Level 2 calorimeter crates) CalTrigger aborts, causing TrigMon to stop.  


<!-- Comment: Thu Nov  2 21;55;14 comment by...Matt -->
For more info on this problem see entries:
Thu Nov 2 14:12:45 and Thu Nov 2 14:53:15
<!-- Date: Wed Nov 1 00:15:48 2000 -->
<hr><p>Shift summary<br> 
Data taken: run 102887 to 102904 (standard).<br> 
What happened:<ul><li>Noisy channels in calorimeter are listed in the e-log<li>Run 102891 had some weird EM calorimeter towers.<li> 
Run 102899 had a bad initialization and a spoiled COT time distribution<li> Run 102904 had a satellite bump, spotted on line: beam loss or playing with the gate ?<li>The last 8 runs were taken with CLC only, so only LuMon could run.</ul>
<!-- Author: S.& S. -->


<!-- Date: Wed Nov 1 00:37:04 2000 -->
<pre>X server on b0dap24 is down after the logout.  
 
The server seems to be down because of the netscape.  
But, we're not sure. You might want to kill the netscape  
before you log out.  
 
> ps -u cdfdaq 
</pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Wed Nov 1 00:37:29 2000 -->
<pre>Koichi and Niccolo is on shift. 
</pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Wed Nov 1 03:12:10 2000 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-10&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-10  width='250' align='top'></td><td>Tracks! </a></td></tr></table>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Wed Nov 1 05:36:18 2000 -->
<pre>Calibration Run 102932: weird cal towers  
- QIE 7 vs ADMem 0 in PEMQIE plot  
- WHAQIE (no power in ADMem>42 ???)</pre>
<!-- Author: Koichi and Niccolo&#39; -->


<!-- Date: Wed Nov 1 08:49:05 2000 -->
Kaori emphasized that the histodisplay for 
YMON and LumMon should be run on B0DAP24 and  
the histodisplay for TrigMon, Xmon and Stage0  
should be run on B0DAP26.   
 
If these histodisplays are run on B0DAP14, it  
will freeze.  YMON is running on B0DAP14 and  
it takes alot of memory to run.
<!-- Author: Olga & Kai -->
<!-- Comment: Wed Nov  1 17;30;52 comment by...Koichi -->
<pre>I recommend not to run netscape on b0dap14 
if you really worry about the memory. 

When you exit netscape, you had better make sure 
that it's not running anymore. I found sometimes netscape 
was running more than one. 


I also recommend not to use the work sheets more than one 
on b0dap14 if you really worry about the memory. 

I'm not expert. Correct me if I'm worng.
</pre>
<!-- Date: Wed Nov 1 09:17:08 2000 -->
Stefano and Saverio have been doing what actually makes sense: 
putting detector related information/plots from YMON, LUMMON, etc 
in the SHIFT e-log instead of the CO E-log, so that people 
will actually read it.  No one (except CO's) reads the CO e-log. 
(someone other than a CO, please correct me if I'm wrong) 
 
  
<!-- Author: Olga -->
<!-- Comment: Wed Nov  1 09;20;37 comment by...Steve Tether -->
<pre>You're wrong!</pre>


<!-- Comment: Wed Nov  1 23;01;45 comment by...Jane -->
TrigMon people read it !
<!-- Comment: Thu Nov  2 09;05;37 comment by...Kaori -->
<pre>I also read CO log, as well.
But any reports which is to do with detector performance, etc..
please report to the main e-log.
Comments/questions specific to the CO operations,
please note them here.

thanks,       </pre>
<!-- Date: Wed Nov 1 14:31:51 2000 -->
<pre>We are trying to make a comment.</pre>
<!-- Author: Bugs Bunny -->
<!-- Comment: Wed Nov  1 14;33;30 comment by...Olga & Kai -->
We learned how to make a comment.




<!-- Date: Thu Nov 2 00:07:49 2000 -->
<pre>Koichi and Niccolo is on shift. 
</pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Thu Nov 2 02:52:11 2000 -->
<pre>Run 103043 shows noise in layer 5 of COT (high voltage is off) 
Also next runs...</pre>
<!-- Author: koichi and niccolo&#39; -->
<!-- Comment: Thu Nov  2 05;00;51 comment by...Koichi -->
<pre>As far as I see in evd, The first appearance was 
103043 event 8662.
</pre>

<!-- Comment: Thu Nov  2 05;01;44 comment by...Koichi , Niccolo -->
<pre>103066. They're gone. But, the new one appear.</pre>



<!-- Date: Thu Nov 2 09:34:20 2000 -->
Talked to Bill Orejudos (COT expert) about DBANA plots for COTCTT calibration. 
<br> 
<br> 
He looked at Run 102934 (see DBANA folder in control room) DBANA plots, and said this could be used as a reference run for the rest of the commissioning run.  The important plots to look at are the offset plots for each crate, offset RMS plots for each crate, and the "slope offset" plot (which is actually occupancy) for each Superlayer (SL).  If nothing drastically changes from run 102934 in these plots, then there isn't need for concern or expert notification.    
<br> 
<br> 
For more information, he just updated the COT calibration documentation.  It can be found from the Ace help page under calibrations. 
<!-- Author: Kai & Olga -->


<!-- Date: Thu Nov 2 11:29:16 2000 -->
<br> 
Does anyone know how to look at Muon calibration data??? 
It does not show up in DBANA!!!! 
 
  
<!-- Author: Olga & Kai -->


<!-- Date: Thu Nov 2 13:27:25 2000 -->
This is the first time I've ever seen the control room quiet.  It's very eery.   
<br>
<!-- Author: Olga -->


<!-- Date: Thu Nov 2 14:12:45 2000 -->
We have tried to restart TrigMon several times and each time, it crashes.   
<br><br> 
***Break*** Segmentation violation 
<br><br> 
Abnormal termination caused by signal: 6 
<br><br> 
 
This seems to be the only error message. 
We cannot seem to run it at all right now!  Help!  
<!-- Author: Kai & Olga -->
<!-- Comment: Thu Nov  2 20;59;37 comment by...Matt -->
Problem was in CalTrigger database access.  After checking for calorimeter crates in the run, we successfully restarted TrigMon with CalTrigger included.  See below for instructions if TrigMon is absolutely crippled by database problems.
<!-- Date: Thu Nov 2 14:23:24 2000 -->
<br> 
Called TrigMon expert, he is coming. 
<br>
<!-- Author: Kai & Olga -->


<!-- Date: Thu Nov 2 14:24:19 2000 -->
<br> 
Commissioning meeting is over.  This is the control room I am used to...complete chaos. 
<br>
<!-- Author: Olga -->


<!-- Date: Thu Nov 2 14:42:56 2000 -->
<br> 
As it says on the muon calibration page, there is no muon calibration consumer, and no data from muon calibrations are being sent to a database.  The data files (in the form of D banks) are looked at by experts.  As CO's, the only thing we should do is make sure the ACE notes which run is a muon calibration run, and then possibly (if time) check to make sure the run (in the form of a data file...not from DBANA) was saved somewhere on fcdfsgi2. 
<br>
<!-- Author: Olga -->


<!-- Date: Thu Nov 2 14:53:15 2000 -->
If you have trouble with TrigMon, here are some pointers from the expert... 
<br> 
<br> 
If you see an error something like: 
<br> 
"stale nfs handle" 
<br>  
then try removing .nfs* files (ie do   rm .nfs*   at unix prompt)  
<br> 
<br> 
If you see errors related to database (trigger table database),  
you should first see if the calorimeter crates are included in the run.  If they aren't included in the run, then it shouldn't work anyway.  If they ARE included in the run,  
then find the .tcl file that TrigMon is running with.  This should be in the output on the top screen of B0DAP24 in the TrigMon window  (at the very start of the output).  In this .tcl file, uncomment the line <br> 
<br> 
module disable CalTriggerDataMaker 
<br> 
and comment out the lines 
<br><br> 
module talk CalTriggerDataMaker 
.<br> 
.<br> 
.<br> 
exit 
<br><br> 
This should fix the database related errors.  
<!-- Author: Kai & Olga -->
<!-- Comment: Thu Nov  2 14;56;01 comment by...Olga -->
the .nfs* files mentioned in this entry will be in the ~cdfdaq directory.

<!-- Comment: Thu Nov  2 20;55;20 comment by...Matt -->
Note: disabling CalTrigger should only be done as a last resort.
Please page TrigMon expert (266-2583) to confirm the problem 
before doing so.  Thanks!










<!-- Date: Fri Nov 3 00:41:50 2000 -->
<pre>Koichi and Niccolo is on shift. 
</pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Fri Nov 3 12:11:15 2000 -->
<pre>We are on shift.</pre>
<!-- Author: kai &  Olga -->


<!-- Date: Fri Nov 3 23:49:34 2000 -->
<br> No collisions but protons are in the ring. 
<br> Most of the crates have been taken by the system experts 
<br> nothing to report. 
<br> SVX still il svx crew hands.
<!-- Author: Stefano and Saverio -->


<!-- Date: Sat Nov 4 00:20:59 2000 -->
<pre>Koichi and Niccolo is on shift. 
</pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Sat Nov 4 05:26:05 2000 -->
<pre>XMon doesn't work. don't know why. Stop XMon.  
</pre>
<!-- Author: Koichi, Niccolo -->
<!-- Comment: Sat Nov  4 14;44;58 comment by...Matt -->
Do you (Koichi & Niccolo) remember what the problem was?  Thanks.
<!-- Comment: Sun Nov  5 01;23;20 comment by...Koichi -->
The segmentation fault, What else could be? <BR>



<!-- Date: Sat Nov 4 10:04:54 2000 -->
<pre>Xmon seems O.K., but has the following error message: 
 
%ERLOG-e SL1D: Total runtime of 5.90807e+08 does not match raw livetime  
        + deadtime sum of 5.90807e+08  4-Nov-2000 10:03:07  
         run = 103456 event = 65747 
 
 
</pre>
<!-- Author: kai & Olga -->
<!-- Comment: Sat Nov  4 14;50;55 comment by...Matt -->
This error message is one of the things that XMon is designed to do.  In the SL1D scaler bank the runtime should always agree with the livetime + deadtime.  In the message above the disagreement is very small: <.0002%.  If the disagreement is large, i.e. you can see it in the numbers reported above, the COs should consult the ace.  Thanks.
<!-- Date: Sat Nov 4 11:35:50 2000 -->
<br> 
Run 103456<br> 
We occasionally got the XMon error message mentioned in the previous entry, but XMon mostly seemed to work fine. 
<br>
<!-- Author: Olga & Kai -->




<!-- Date: Sat Nov 4 15:56:30 2000 -->
<br> 
Vaia Papadimitrious came by and said that it would also be helpful to print pedestal plots from DBANA when a QIE calibration is done.  This gives a little more information than just the QIE plots.  The option to plot pedestals is in DBANA as CEMPED, CHAPED, PEMPED, PHAPED, WHAPED under the Detectors menu. 
<br>
<!-- Author: Olga -->


<!-- Date: Sat Nov 4 18:10:16 2000 -->
<pre>Collisions.  
LuMon crashed with segmentation violation error.  
Re-started</pre>
<!-- Author: Stefano and Saverio -->


<!-- Date: Sat Nov 4 18:59:28 2000 -->
<pre>LuMon crashed. 
Segmentation violation 
</pre>
<!-- Author: Stefano & Saverio -->


<!-- Date: Sat Nov 4 19:00:13 2000 -->
<pre>Also Stage0 crashed. 
</pre>
<!-- Author: S & S -->


<!-- Date: Sat Nov 4 19:12:16 2000 -->
<pre>Tracks have disappeared from the Event Display, can't make them come back.</pre>
<!-- Author: S & S  -->


<!-- Date: Sat Nov 4 21:04:34 2000 -->
<p>Run 103532<ul><li>CLC is not in the data stream. 
<li> ISLD banks have an error: out-of-bounds exception, 
Event Display crashes due to this.</ul>  
<!-- Author: Stefano and Saverio -->


<!-- Date: Sat Nov 4 21:35:01 2000 -->
<pre>ISLD banks dropped from consumers.</pre>
<!-- Author: Koichi, Stefano, Saverio -->


<!-- Date: Sat Nov 4 23:30:04 2000 -->
<pre>Stage0 consumer error message is repeated continuously:  
ERROR: Socket to display server is no longer valid. 
Consumer stopped and restarted during run 103549 
Error Message disappeared.</pre>
<!-- Author: Stefano and Saverio -->


<!-- Date: Sun Nov 5 00:15:02 2000 -->
<pre>Koichi and Niccolo is on shift. 
</pre>
<!-- Author: Koichi, Niccolo -->



<!-- Date: Sun Nov 5 05:49:07 2000 -->
<pre>No tracks in the online evd.  
But, tracks in the offline 3.11.0 evd. </pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Sun Nov 5 09:09:06 2000 -->
We see many clear tracks on online Event Display.  
<!-- Author: Olga -->


<!-- Date: Sun Nov 5 09:42:19 2000 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-13&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-13  width='250' align='top'></td><td>fired bits: 46 event</a></td></tr></table>
<!-- Author: O.K.  -->


<!-- Date: Sun Nov 5 11:24:02 2000 -->
<br> 
Consumers seem to be running (the displays on B0DAP24 are running), but when you go to the Connection menu and look under status, there are no connections listed.  Also on the consumer status page, there are no connections listed.  But if we just guess which machine and which port the consumers are running on, and we happen to guess right, we can still look at plots.   
<br>  
<!-- Author: O. K. (Olga & Kai) -->


<!-- Date: Sun Nov 5 14:56:40 2000 -->
<br> 
<br> 
Hits and tracks disappeared from event display.  Does anyone know how to make them come back?  Stefano and Saverio had the same problem last night...did you figure out how to make them come back? 
<br> 
<br>
<!-- Author: O. K.  -->


<!-- Date: Sun Nov 5 15:20:30 2000 -->
<br> 
<br> 
Our all-knowing ACE showed us how to get hits in the COT back on Event Display: 
There is a little Correction box in the COT cuts area on the event display GUI.  If the Correction box is checked, and there is a t0 or LE cut, then hits will reappear.  We don't see tracks because tracking isn't being included in L3 reconstruction right now.  (run 103575) 
<br> 
<br>
<!-- Author: O. K. -->


<!-- Date: Sun Nov 5 15:40:24 2000 -->
<br> 
<br> 
We still have collisions.  They are scheduled to last until Monday morning at 7am.  If there is a problem with the beam and it is for some reason lost, could the evening shift please send mail to the owl shift if they do not have to come in? 
<br> 
<br> 
  
<!-- Author: O. K. -->


<!-- Date: Sun Nov 5 20:37:22 2000 -->
LuMon Crashed 
<!-- Author: {Quick_Entry} -->


<!-- Date: Sun Nov 5 22:45:38 2000 -->
<pre>run number did not change in all the consumer monitors 
between 103595 and 596. Monitors re-started</pre>
<!-- Author: S&S -->


<!-- Date: Mon Nov 6 00:15:00 2000 -->
<pre>Koichi and Niccolo is on shift. 
</pre>
<!-- Author: Koichi, Niccolo -->


<!-- Date: Mon Nov 6 00:30:38 2000 -->
<pre>There seems to have some trouble with  
exiting the netscape. It still stays in the process 
even if the window is disappeared. Investigate. 
May be, something wrong with the resource  
I changed before. </pre>
<!-- Author: Koichi -->
<!-- Comment: Mon Nov  6 03;36;10 comment by...Koichi -->
I believe, a resource has nothing to do with. 
don't know why. 



<!-- Comment: Mon Nov  6 06;40;44 comment by...Koichi -->
It seems it takes some time before <BR>
it really exits from the process. I can't believe.

<!-- Comment: Mon Nov  6 10;39;08 comment by...KOichi -->
By the way, I modified the resource file ~/Netscape so as not 
to depend on the netscape version. 
<!-- Date: Mon Nov 6 05:34:23 2000 -->
<pre>can't lookup the offline data. don't know why. 
</pre>
<!-- Author: Koichi -->


<!-- Date: Mon Nov 6 06:37:08 2000 -->
<pre>We are going home!  
</pre>
<!-- Author: Koichi, Niccol -->



<!-- Date: Mon Nov 6 10:32:08 2000 -->
<pre>We are of shift.</pre>
<!-- Author: Olga & Kai (O. K.) -->
<!-- Comment: Mon Nov  6 14;42;19 comment by...kai -->
We are on shift


<!-- Date: Mon Nov 6 15:30:24 2000 -->
<br> 
We are trying to run ProductionExe on some files that Young Kee wanted us to run on.  So far, we have not been able to run on more than 43 events without things crashing.  Things were very very very slow (~3 minutes or more per event).  Talked with Liz Sexton-Kennedy to try and figure out a way to make things run without crashing and also run faster. 
<br> 
<br> 
The fix was to put the following lines at the end of  
Production/setup_input.tcl 
<br><br> 
talk PVFinder<br> 
   TrueVertices set f<br> 
exit<br> 
<br> 
module disable SiPatternRecModule 
 
When we did this, things ran much much much much faster (100 events in ~10 minutes) 
  
<!-- Author: Olga -->
<!-- Comment: Mon Nov  6 15;31;11 comment by...Olga & Kai -->
We succesfully ran on 100 events...we'll now try many more events (10000)


<!-- Date: Mon Nov 6 15:44:40 2000 -->
<pre>As of today, the entire procedure for running ProductionExe is as follows: 
 
First log into your account on fcdfsgi2 (if you are an online  
machine in the control room, you have to first ssh to cdfsga  
and from there ssh to fcdfsgi2)  
 
Then do the following:  
 
source ~cdfsoft/cdf2.cshrc  
setup cdfsoft2 development  
newrel -t development test  
cd test  
addpkg -h Production  
 
cp /cdf/data04/s3/commission/doc/setup_input.tcl Production/.  
cp /cdf/data04/s3/commission/doc/setup_output.tcl Production/.  
 
After copying (doing the cp commands above) the .tcl files, you  
need to look in setup_input.tcl and change the input file to the  
particular run you want to run production on.  
 
As of today (in order to prevent crashing) you need to add the following lines to the end of setup_input.tcl 
 
talk PVFinder 
   TrueVertices set f 
exit 
 
module disable SiPatternRecModule 
 
 
Then, you also need to change a line in setup_output.tcl file.  Change 
 
  output paths     AA AllPath 
 
to 
 
  output paths     AA Production 
 
Add to run_Production.tcl just before "exit" 
 
begin -nev xxx 
where xxx is the number of events you want to process. 
 
Then do  
ProductionExe Production/run_Production.tcl >& Production.output 
 
 
This should do the trick.</pre>
<!-- Author: Olga & Kai (O.K.) -->


<!-- Date: Fri Jan 19 17:41:45 2001 -->
<pre>Consumer icon too smal for event display 
We had to use rawhide 
</pre>
<!-- Author: Saverio -->


<!-- Date: Thu Feb 22 09:37:32 2001 -->
<pre><p>If you need to compile code on shift (e.g. to run the offline validation code) you should use b0dap30 (which has a license) to compile and then you can run on any of the local machines.</pre>
<!-- Author: Napora -->


<!-- Date: Wed Mar 14 08:50:41 2001 -->
<pre>test</pre>
<!-- Author: Matt -->


<!-- Date: Sun Mar 18 04:16:40 2001 -->
A tip for DBANA: Whenever you use DBANA, don't mix with a process 
which requires "setup fer" (like DAQMon).
<!-- Author: Un-ki -->


<!-- Date: Sat Mar 31 10:16:33 2001 -->
<pre>The time on the YMON display seems to be connected to 
the run start time. Isn't supposed to update as time passes by? </pre>
<!-- Author: fotis -->


<!-- Date: Tue Apr 10 00:02:51 2001 -->
<pre>There seem to be some inconsistent features in the CES/CPR/CCR 
data. I can see which wedges are read out for various  
consumers and see the following: 
  (1) YMON: CES/CPR/CCR W9-17, 35-43 show up in occupancy plots. 
  (2) CESPED: CES W12-26, 15 total wedges 960 wire/1920 strip 
      entries (from DBANA). 
      CCRPED: Cannot tell which wedges, but see 240 entries, 
          therefore 24 wedges (10 CCR per wedge). 
      CPRPED: Cannot tell which wedges, but see 768 entries, 
          therefore 24 wedges (32 CPR per wedge). 
     
      Thus, CESPED is missing 9 wedges compared with CCR/CPR. 
  
   (3) CESQIE: CES W12-21+24-26 (22-23 dead) and W39-47. 
           total 22 wedges, 1408 wire/2816 strip entries. 
       CPRQIE, CCRQIE entries in "SloErr/Slope Mean by QIE.." 
           make no sense, so cannot use this. 
  
       Thus, CESQIE has 9 extra wedges compared with CESPED 
       (ignoring the fact that W22-23 is missing in CESQIE but  
        present in CESPED).  Adding these to CESPED would 
       give it 24 wedges, same as CCR/CPRPED.  </pre>
<!-- Author: wicklund -->


<!-- Date: Tue Apr 10 00:12:52 2001 -->
<pre>The ./startConsumers.sh ID works though the 
script was edited to run xmon and trigmon both 
on b0dap55. Use this version for now.</pre>
<!-- Author: wicklund -->


<!-- Date: Mon Apr 16 10:07:53 2001 -->
<pre>Kaori has updated cdfsoftb0 and the script startConsumers.sh. 
Now there are 8 consumers started by default.</pre>
<!-- Author: Beate -->


<!-- Date: Mon Apr 23 15:14:18 2001 -->
<pre>I've updated and reorganised the instructions for the CO: 
/cohelp/cohelp.html 
Hopefully its a bit easier to find out what_to_do_on_shift. 
</pre>
<!-- Author: Ronan -->


<!-- Date: Mon Apr 23 15:44:48 2001 -->
 
There is a new way of starting one or all of the consumers.<br> 
Instead of <tt> ./startConsumers.sh `partition number' </tt><br> 
type <tt> ./startMon.sh `Monitor type' `partition number'</tt> 
 
<br><br> 
Check the <a href=/ops/cohelp/cohelp.html> 
CO pages </a> for more info. 
 
<br><br> 
(By the way, the old command still works.)
<!-- Author: Ronan -->


<!-- Date: Thu Apr 26 09:22:04 2001 -->
There is a new method for starting all the consumer displays. 
Type<br> 
<tt> ./startDisplay &lt monitor &gt </tt><br> 
where &lt monitor &gt is Xmon, YMon etc. or All (to start all 
displays). 
<p> 
See  
<a href=/ops/cohelp/how_to_histodisplay.html> 
the relevant CO help page </a> 
for more instructions on the updated procedure. 
 
  
<!-- Author: Ronan -->


<!-- Date: Fri Apr 27 11:21:23 2001 -->
TrigMon is having problems with CalTriggerDataMaker (TrigSim) this morning.  Currently running with CalTriggerDataMaker disabled in tcl file, which means that the DIRAC/DCAS sim vs data plots will make no sense.  About 700 events went into TrigMon113441al.root, all the other TrigMon root files from this run were created while we were debugging and should be junk.
<!-- Author: Matt -->


<!-- Date: Fri Apr 27 15:49:02 2001 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Fri_Apr_27_2001_154844.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-17  align='top'></td><td>36 on 36 x versus y</a></td></tr></table>
<!-- Author: Pete Renton -->


<!-- Date: Fri Apr 27 15:58:29 2001 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-19&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-19  width='300' align='top'></td><td>x versus y (correct orientation)</a></td></tr></table>
<!-- Author: Pete Renton -->


<!-- Date: Fri Apr 27 16:05:28 2001 -->
This afternoon we fixed a database access bug in XMon that was preventing XMon from reading the Trigger DB and crashing XMon.  This fix is not in the frozen cdfsoftb0 release, so for now I have taken over running XMon from a private release.  Unless asked to by myself, COs should not be running XMon!  Thanks.
<!-- Author: Matt -->
<!-- Comment: Fri Apr 27 16;37;32 comment by...Matt -->
Please avoid using sophisticated HTML codes in comments. (click to clear this message)



<!-- Date: Fri Apr 27 16:39:02 2001 -->
I am also running TrigMon from my private area.  I will inform COs when I am finished.
<!-- Author: Matt -->


<!-- Date: Fri Apr 27 16:48:04 2001 -->
<pre>BeamMon crashed,  sent mail to Hartmut with the error message. 
</pre>
<!-- Author: Vladimir Rekovic -->


<!-- Date: Fri Apr 27 19:46:25 2001 -->
<pre>Could not start LumMon.  
It did not start together with other conumers when entered command 
  &gt;./startMuon All 4 
 
It did not start when entered command 
  &gt;./startMon LumMon 4  
we got 
 
This script should be executed by a Consumer Operator 
  
./startMon.sh: [-e: command not found 
starting LumMon partition 4 on b0dap04  
  
to view the results of consumer monitors, 
go to a terminal where you like to display, 
open a window, then type: 
  
source ~cdfsoft/cdf2.cshrc 
setup cdfsoftb0 
HistoDisplayMain   
 
 
  
</pre>
<!-- Author: Muge Karoz, Vladimir Rekovic -->


<!-- Date: Fri Apr 27 19:53:28 2001 -->
<pre>xmon crashed, we restarted it and is working now..</pre>
<!-- Author: muge, vladmir -->


<!-- Date: Fri Apr 27 21:46:52 2001 -->
<pre>Song Wing Wang fixed the problme with LumMon.it is  
up and running now.</pre>
<!-- Author: muge karagoz -->


<!-- Date: Sat Apr 28 00:25:27 2001 -->
I am leaving my development version of TrigMon and XMon running on b0dap55 for the night.  If (when) they crash the COs should start the production (cdfsoftb0) versions on any subsequent runs.  The development version of TrigMon contains new code (esp. from SVT) so it is preferable not to kill these and start new consumer processes.
<!-- Author: Matt -->


<!-- Date: Sat Apr 28 14:59:51 2001 -->
<pre>14:57 SiliMon crash (2nd time) 
 
         run = 113500 event = 437742 
%ERLOG-e ROOT/TFile:  
         file /data1/consumer/results/SiliMon/SiliMon113500ab.root already exists  
         Framework TFile::TFile() 28-Apr-2001 14:55:47  
         run = 113500 event = 437742 
SiliMonModule: Root file /data1/consumer/results/SiliMon/SiliMon113500ac.root saved. 
SiliMonModule: summary file /data1/consumer/results/SiliMon/SiliMon113500.sum written. 
DisplayServer: INFO: SiliMon sent message: END CONNECTION 
               ==&gt; Going to terminate connection to SiliMon running on 
               b0dap67.fnal.gov 
</pre>
<!-- Author: Pete Renton -->


<!-- Date: Sat Apr 28 15:46:08 2001 -->
<pre>SiliMon crashed again; restart</pre>
<!-- Author: Pete -->


<!-- Date: Sat Apr 28 17:37:50 2001 -->
during run 113508, the consumers were not able to receive any events from the csl, even though the csl was receiving data. 
the csl experts were paged.
<!-- Author: muge -->


<!-- Date: Sat Apr 28 20:37:53 2001 -->
<pre>all consumers are running ok _now_ for the run 113517 
(in regards to previous log entry..)</pre>
<!-- Author: muge -->


<!-- Date: Sun Apr 29 01:12:21 2001 -->
<pre>All consumers are running ok, altough LumMon error-output terminal does not appear. 
 
A strange behaviour in COT SL6 was found: In YMON, SlideCOTStart, COT Start Time for SL6 histogram looks rather different to the other 
SLs in the same page:channels between 0 and 100 are very much higher than the rest.  It was reported 
to the SciCo</pre>
<!-- Author: Javier Fernandez -->


<!-- Date: Sun Apr 29 10:14:21 2001 -->
<pre>Request for CO's from XMon experts:  
         please print out XMon summary 
         once/hour when we are running.</pre>
<!-- Author: Pete -->
<!-- Comment: Sun Apr 29 10;27;06 comment by...Pete -->
please also e-mail these to Jonathon Lewis (jdl@fnal.gov)

<!-- Comment: Sun Apr 29 11;12;25 comment by...Matt -->
To print out XMon summaries, go to the XMon results area: /data1/consumer/results/XMon.  Each run will have 2 text files here, one ending in .dat and one in .log.  The COs should print out the .dat file (not the .log file!) which should be 1-2 pages of text.  The hard copies should be kept in the Consumer Monitor Reference Histogram binder.  I have printed out the summaries through run 113541, bringing the binder up to date as of this entry.


<!-- Date: Sun Apr 29 13:02:45 2001 -->
<pre>Trigmon crashed 12:46</pre>
<!-- Author: Pete -->


<!-- Date: Sun Apr 29 18:33:49 2001 -->
Xmon has been crashed.it is running now but it missed the beginning of run 113555.
<!-- Author: muge -->


<!-- Date: Sun Apr 29 19:28:50 2001 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-20&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-20  width='300' align='top'></td><td>run 113555</a></td></tr></table>
<!-- Author: muge -->
<!-- Comment: Sun Apr 29 19;33;51 comment by...muge -->

noises in shower max?. ace is reported. no idea how this looked like previously so i have placed the plot above..


<!-- Date: Sun Apr 29 20:29:36 2001 -->
SiliMon was reporting chip status errors. it crashed while Jean and I was looking at error messages. however, Silimon cannot find the .tcl file and cannot start. all is also in Silicon e-log. will call/email Kaori Maeshima about it.
<!-- Author: muge -->


<!-- Date: Sun Apr 29 21:06:50 2001 -->
<pre>Hartmut gave instructions for running SiliMon temprorarily. 
Until back to normal, the Co's should run them by issuing these 
commands (under the machine where it has already been running, 
for the moment it is b0dap67): 
 
source ~cdfsoft/cdf2.cshr 
setup cdfsoftb0 
runSiliMon &lt;partition number&gt; 
 
this should make SiliMon work..</pre>
<!-- Author: muge  -->
<!-- Comment: Mon Apr 30 08;31;32 comment by...kaori -->
type: source ~cdfsoft/cdf2.cshrc  (not 'cshr')

<!-- Date: Sun Apr 29 21:09:10 2001 -->
<pre>XMon crashed and now it is restarted.</pre>
<!-- Author: muge -->


<!-- Date: Sun Apr 29 22:20:49 2001 -->
clc qie and tof calibs do not show up in the dbana. aces are noted. they will retake those.
<!-- Author: muge -->


<!-- Date: Mon Apr 30 00:31:10 2001 -->
<pre>XMon crashed, restarted and run its display</pre>
<!-- Author: Javier -->



<!-- Date: Mon Apr 30 10:23:56 2001 -->
<pre>Cant open displays on b0dap56; get ... 
 
/usr/bin/X11/xauth:  creating new authority file /tmp/Xauth5430_29259 
access control disabled, clients can connect from any host 
/usr/bin/X11/xauth:  creating new authority file /tmp/Xauth5430_29320 
Starting HistoDisplayMain on b0dap56 
b0dap51:57:~&gt;/usr/bin/X11/xauth:  creating new authority file /tmp/Xauth5430_29383 
Xlib: connection to "b0dap56.fnal.gov:0.0" refused by server 
Xlib: Client is not authorized to connect to Server 
/usr/X11R6/bin/xterm Xt error: Can't open display: b0dap56.fnal.gov:0.                             
 
Kaori informed and will investigate.</pre>
<!-- Author: Pete -->


<!-- Date: Tue May 1 02:10:54 2001 -->
<pre>All monitors and Event Display were restarted in partition 0</pre>
<!-- Author: Javier -->


<!-- Date: Tue May 1 02:27:55 2001 -->
<pre>Histograms are normal for this kind of trigger</pre>
<!-- Author: Javier -->


<!-- Date: Tue May 1 16:41:24 2001 -->
<pre>a CMU occupancy plot for the cosmic ray run 113649 is saved  
under cdfdaq home dir (CMU2dOccupancyCanvas.ps) upon Hyunsoo's 
 request. </pre>
<!-- Author: muge -->


<!-- Date: Tue May 1 17:07:16 2001 -->
<pre>if a CO is requested to look at previous runs' consumer outputs offline, is there already a release in cdfdaq home area 
for the Co's to use?</pre>
<!-- Author: muge -->


<!-- Date: Tue May 1 18:26:48 2001 -->
trigmon crashed.error message is: 
 
 
 *** Break *** floating point exception 
%ERLOG-s Abnormal termination: caused by signal:  6 Framework abortJob()  
          1-May-2001 18:22:52 run = 113651 event = 0 
 XFTTrigger aborting !  
 
 *** Break *** segmentation violation 
Display Server: End with signal 11 ( = SIGSEGV ) 
Report end to the StateManager 
sockets closed  
<!-- Author: muge -->


<!-- Date: Tue May 1 20:53:31 2001 -->
 
upon request of D. Amidei, an offline trigmon is run to  
analyze run 113610 of last night's. he will be informed  
at the shift turn.
<!-- Author: muge -->


<!-- Date: Wed May 2 00:54:19 2001 -->
<pre>Event Display in b0dap50 apparently hang up with the finish of the run, I waited for a recovery but it did not so I restarted again it</pre>
<!-- Author: Javier -->


<!-- Date: Wed May 2 01:28:34 2001 -->
<pre>Monitors barely survived after the new run beginning, all were restarted again.</pre>
<!-- Author: Javier -->


<!-- Date: Wed May 2 01:31:14 2001 -->
Hot Towers detected in TrigMon for run 113657:
<!-- Author: Javier -->


<!-- Date: Wed May 2 01:33:56 2001 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Wed_May__2_2001_013342.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-22  align='top'></td><td>After capturing this plot using "Save As" option, TrigMon crashed giving the following error:   *** Break *** floating point exception %ERLOG-s Abnormal termination: caused by signal:  6 Framework abortJob()            2-May-2001 01:23:42 run = 113657 event = 0  XFTTrigger aborting !    *** Break *** segmentation violation Display Server: End with signal 11 ( = SIGSEGV ) Report end to the StateManager Press C-d to close window. sockets closed </a></td></tr></table>
<!-- Author: Javier -->


<!-- Date: Wed May 2 01:39:38 2001 -->
<pre>So..., TrigMon was restarted along with its Dsiplay</pre>
<!-- Author: Javier -->


<!-- Date: Wed May 2 12:35:23 2001 -->
<pre>Unable to restart ObjectMon after it crashed... 
 
b0dap51:23:~&gt;aklog: can't get afs configuration (afsconf_Open(/usr/vice/etc)) 
/usr/bin/X11/xauth:  creating new authority file /tmp/Xauth4958_27557           </pre>
<!-- Author: Pete -->


<!-- Date: Wed May 2 18:43:12 2001 -->
i killed the already running mon's and started x, y, and trig mon's afresh.  
the reason: they were up the whole day without the db being up and i thought it's a good idea to initialize them again.  
the question: is this really a good idea?how important is the initialization for the consumers reagrding the db?
<!-- Author: muge -->


<!-- Date: Wed May 2 22:04:22 2001 -->
2 notes:<br> 
1. in trigmon, dirac sum et comparisons look very different from the reference plots, however Aaron says this trigger is not included in this run (113701), so it should be ok.<br> 
2. xmon is not showing the live and dead times correctly(infact, no output at all)?will reset it...  
<!-- Author: muge -->
<!-- Comment: Wed May  2 22;41;33 comment by...muge -->
sorry, it's dcas not dirac.. but it's still weird for me.more notes in general e-log..
<!-- Date: Wed May 2 22:16:51 2001 -->
 
could there be something wrong with the trigger db? xmon restarted saying "no connection to trigger db" and is still showing no live dead times... as i understand, the trigger rate monitor is not working as well.. (run 113701)
<!-- Author: muge -->



<!-- Date: Thu May 3 00:26:36 2001 -->
<pre>A TrigMon was running on b0dap55 since very long time, it was using much of the system resources and slowing down the rest of processes (even the mouse!!); since no Display and konsole for this TrigMon were present, it was killed. Now the system seems normal to run the real Monitors with the new run about to start</pre>
<!-- Author: Javier -->


<!-- Date: Thu May 3 01:16:57 2001 -->
<pre>Same behaviour as reported by the previous CO: XMon histograms are empty at all although the db and socket seem ok and the Number of Bad Diracs per event histogram in TrigMon is greater than expected.</pre>
<!-- Author: Javier -->


<!-- Date: Thu May 3 12:00:11 2001 -->
<pre>b0dap55 stuck; rebooted.</pre>
<!-- Author: Pete -->


<!-- Date: Mon Jun 4 19:37:41 2001 -->
<pre>Some COT TDCs (superlayer 4) were taken out. 
The corresponding occupancy plots will be empty around cell 250. 
</pre>
<!-- Author: Mauro -->


<!-- Date: Tue Jun 5 11:11:14 2001 -->
<pre>With New DBANA in cdfsoftb0 (since 5.Jun), 
LED/XEF histos looks red because of using transparent Canvas. 
Please Print them out. You can see better. 
</pre>
<!-- Author: Isamu -->


<!-- Date: Tue Jun 5 18:56:05 2001 -->
<pre>A second TrigMon process appears from time to time on the consumer status page, 
and later it disappears. 
It runs on b0dap66 and apparently does not interfere with main TrigMon. 
What are the instructions in this case ?  What should the CO do ? 
</pre>
<!-- Author: Mauro -->


<!-- Date: Tue Jun 5 19:08:57 2001 -->
<pre>It turns out that the second TrigMon process is run by the trigger people. 
It would be nice to have the partition number displayed on the Consumer Status page. 
</pre>
<!-- Author: Mauro -->
<!-- Comment: Wed Jun  6 18;18;18 comment by...Kaori -->
<pre>Currently, the consumer status manager process does not have
the partition number information, therefore,
the partition numbers are not displayed. Unfortunately, Getting
the partition number information
into the status manager is not a very simple modification.
However, we can still tell which TrigMon is the official one from
the status page, because the 'official' TrigMon should be running
on b0dap52. (see 'how_to_start_consumers' web page).
You might notice that, time to time, we may run other monitors
(not only TrigMon) for the test purpose, but they should all run
on the different machine from the officially assigned node.</pre>
<!-- Date: Wed Jun 6 17:27:54 2001 -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>To all Consumer Operators: 
 
Scripts for starting consumer monitors and display were moved to: 
~cdfdaq/consumer_scripts</pre></td></tr></td></table>
<!-- Author: Mauro -->


<!-- Date: Wed Jun 6 18:16:53 2001 -->
<pre>Kaori asked to send her an e-mail (maeshima@fnal.gov) each time a 
consumer monitor makes errors or crashes during normal operations. 
Please document the error message in the e-mail. 
</pre>
<!-- Author: Mauro -->
<!-- Comment: Wed Jun  6 18;21;53 comment by...Kaori -->
<pre>Yes, please cut-paste the 'error messages' when things
crash and send to me...  (thanks!)
By the way, we know that LumMon crashes at very beginning.
We are working on it!</pre>


<!-- Date: Wed Jun 6 18:27:35 2001 -->
<pre>The new HistoDisplay script starts up with all the consumers 
attatched to the proper HistoDisplay GUI for you. 
But if the consumer is not running (not showing on the consumer 
status page (as 'running'), or more than two same name consumers 
are running, it will give you a warning. 
Again, please note that we have moved start-up scripts to 
~cdfdaq/consumer_scripts area.</pre>
<!-- Author: Kaori -->


<!-- Date: Thu Jun 7 21:57:01 2001 -->
<pre>When a run ends and a new run starts, if LumMon continues to run across run boundaries 
his histograms get negative entries. 
This might even be a feature rather than a bug, but the CO must be aware of it. 
</pre>
<!-- Author: Mauro -->


<!-- Date: Sun Jun 10 02:18:28 2001 -->
<pre>We saw a big spike in the YMon Central Muon Slide, 
it is in the plot CMP Ocupancy for part S, around stack 30   
run #117303, reported to aces 
 
Yi Le </pre>
<!-- Author: Yi Le -->


<!-- Date: Sun Jun 10 02:48:59 2001 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-28&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-28  width='300' align='top'></td><td>plot of CMP occupancy for part S in YMon run #117303</a></td></tr></table>
<!-- Author: Yi Le  -->


<!-- Date: Sun Jun 10 07:42:07 2001 -->
<pre>The spike we saw above disappeares in run# 117315 
everything looks normal now 
</pre>
<!-- Author: Yi Le -->


<!-- Date: Sun Jun 10 14:25:13 2001 -->
<pre>Date: Sun, 10 Jun 2001 15:05:57 -0400 (EDT) 
From: Arie Bodek &lt;bodek@pas.rochester.edu&gt; 
To: vaia@fnal.gov, willis@fnal.gov, hbudd@fnal.gov, tesarek@fnal.gov, 
    robine@fnal.gov 
Cc: Arie Bodek &lt;bodek@pas.rochester.edu&gt; 
Subject: Calorimetery Pedestal Checks 
To: Vaia Papadimitriou, Willis Sakumoto, Howard Budd 
  
From: Arie Bodek 
  
Copy: Rick Tesarek, Robin Erbacher, Willis Sakumoto 
  
Subject: Calorimeter DBANA plots and checks 
  
There appears to be some real problems and some possible fictitious problems iwth the pedestals. Robin Erbacher has mentioned to me that a CO should look at wide pedestals to look for bad channes. 
  
I looked at wide pedestals for the PHA and on two consequite days, the channels with RMS pedestals of 8 are not the same. 
  
There may be bad channels there, but if they are not bad two days in a row, it may mean that we have one bad read that screws up the RMS for a particular channel.  I do not know if this is the real reason, and I could not use DBANA to plot the pedestals for the channels that got a wide RMS, so I could not know why it was wide. If a check shows that ONE bad read is the cause, then one make another plot which is the restricted Mean and RMS of the pedestals, where the Mean and RMS are only calculated for a restricted range. (say between 100 and 300 with the overflows/underfolows counter to see how many bad reads there are). In general, if the regular RMS is bad but the restricted RMS  is good, than it may imply a few bad reads. If the fraction of bad reads is large, it implies a different problem. 
  
2.  The QIE calibrations and pedestals for the WHA look bad 
say for run 117328.  It looks here like  a real hardwar problem. 
there are some pedestals with 50 and 0 instead of 170 values. 
Also the QIE slopes have 3 peaks. 
             </pre>
<!-- Author: Arie Bodek -->


<!-- Date: Mon Jun 11 11:15:42 2001 -->
<pre>Date: Mon, 11 Jun 2001 12:11:22 -0400 (EDT) 
From: Arie Bodek &lt;bodek@pas.rochester.edu&gt; 
To: oldeman@fnal.gov, vila@fnal.gov 
Cc: tesarek@fnal.gov, Maeshima@fnal.gov, vaia@fnal.gov 
Subject: TOF DBANA and WWW documentation 
   
Greetings to the TOF group 
  
I am the CO on day shift this week. The DBANA plots for the TOF 
appear to be labeled WCal.  This tends to cause confusion, because I sent messages to the Wall Calorimeter people telling them about the TOF problems. 
  
(1)  Can you  label the TOF plots and TOF please. 
  
(2) Netscape from the Control room computers crashes if one tries to click on a link that tries to send Email from Nescape. It would be nice if all WWW documentation has both the name of the person to contact and the Email address shown on the WWW page (and also linked to the Email address, when they get around to making the netscape in the control room work). This way at least people can use Pine and send Email, without having to go the FNAL phone book to find the correct Email address. 
  
(3) Can the WWW consumer page on calibration be updated to show 
examples of the correct plots. It is also out of date. for example  DBANA in development does not work and it crashes, so it must use the standard DBANA & 
  
(4) Now the TOF plots seem to have pedestals at 0, 50 , 175. It 
may be that some of the ADC's have the wrong switches. I assume that all pedestals should be at 175. 
  
(5) The QIE calibrations also have 3 peaks in them at 0.15, 0.22 and 0.35 for the error/slope          </pre>
<!-- Author: Arie Bodek -->


<!-- Date: Tue Jun 12 04:41:18 2001 -->
<pre>--Can't start SiliMon 
 
There are lots of error messages as following in SiliMon, 
I try to restart it, the situation keeps same, and consumer 
display can't find SiliMon 
 
%ERLOG-e BANKS-Data Write Out-of-Bounds: offset =  0 , index =  468353  
        , data_size =  16689 , value =  -22968 , Bank ( SIXD ,  1 )  
         GlobalLibraryLogger StorableBank::set_I2_element()  
         12-Jun-2001 04:37:47 run = 117491 event = 80961 
%ERLOG-e BANKS-Data Write Out-of-Bounds: offset =  0 , index =  468354  
        , data_size =  16689 , value =  -23224 , Bank ( SIXD ,  1 )  
         GlobalLibraryLogger StorableBank::set_I2_element()  
         12-Jun-2001 04:37:47 run = 117491 event = 80961</pre>
<!-- Author: Yi Le -->
<!-- Comment: Tue Jun 12 04;44;53 comment by...Yi Le -->
<pre>I tried to restart it several times, once it gave following
error messages

 
         SUB=@SiClusterningModule::event NOT_CHIP STRIP_ORDER
CHIP_MISMATCH
         NO_EOF CHIP_INVALID CHIP_COUNT UNKNOWN_WORD CHIP_SIDEERROR
         SiClusteringModule 12-Jun-2001 04:17:50 run = 117491 event =
43836 %ERLOG-s Readout errors detected in Silicon Raw Banks (SIXD, ISLD):
         SUB=@SiClusterningModule::event NOT_CHIP STRIP_ORDER
CHIP_MISMATCH
         NO_EOF CHIP_INVALID CHIP_COUNT UNKNOWN_WORD CHIP_SIDEERROR
         SiClusteringModule 12-Jun-2001 04:17:50 run = 117491 event =
43836 out_of_range: Out of range exception occured. %ERLOG-s Abnormal
termination: caused by signal:  6 Framework abortJob()
         12-Jun-2001 04:17:57 run = 117491 event = 43836 %ERLOG-s Abnormal
termination: caused by signal:  6 Framework abortJob()
         12-Jun-2001 04:17:57 run = 117491 event = 43836
 F: Dummy implementation of old stager API called F: Dummy implementation
of old stager API called SiliMonModule: Root file
/data1/consumer/results/SiliMon/SiliMon117491aa.root saved. SiliMonModule:
summary file /data1/consumer/results/SiliMon/SiliMon117491.sum written.
DisplayServer: INFO: Consumer sent message: END CONNECTION
               ==&gt; Going to terminate connection to Consumer running on
               b0dap67.fnal.gov   </pre>


<!-- Date: Tue Jun 12 06:22:30 2001 -->
<pre>Can't open display on b0dap55 anymore 
 
 /usr/bin/X11/xauth:  creating new authority file /tmp/Xauth5430_22808 
Xlib: connection to "b0dap55.fnal.gov:0.0" refused by server 
Xlib: Client is not authorized to connect to Server 
/usr/X11R6/bin/xterm Xt error: Can't open display: b0dap55.fnal.gov:0. 
 
sent email to Kaori  
  
 
 
</pre>
<!-- Author: Yi Le -->
<!-- Comment: Tue Jun 12 06;40;08 comment by...Yi Le -->
<pre>solved this problem by type
xhost + b0dap51.fnal.gov  on b0dap55
</pre>


<!-- Comment: Tue Jun 12 08;02;53 comment by...Yi -->
<pre>after talking to kaori, above solution might not be true,
it is because of memory leak 
kaori suggests we stop all processed including both monitor and display processes, then restart them
</pre>
<!-- Date: Tue Jun 12 07:59:51 2001 -->
<pre>SiliMon can run now, I don't know why, but this run 11746 
is a cosmic run,</pre>
<!-- Author: Yi Le -->
<!-- Comment: Tue Jun 12 08;09;04 comment by...Yi Le -->
<pre>run # is 117496</pre>


<!-- Date: Fri Jun 15 13:20:12 2001 -->
<pre>  
Date: Fri, 15 Jun 2001 13:05:59 -0500 (CDT) 
From: Masa Tanaka &lt;masa@hep.anl.gov&gt; 
To: tesarek@fnal.gov, robine@fnal.gov 
Cc: bodek@fnal.gov, Masa Tanaka &lt;masa@hep.anl.gov&gt; 
Subject: Calibration issues for Calorimeters 
   
 
  
 Looking at the calibration runs of 117824 (6/15/01) and 
117721 (6/14) for common problems of both runs. 
QIE calibration: 
* CEM  wedge-7  left  east 0 
            6  left  east -1 
            15 right west 0 
            6  right east -8 
  
* CHA  wedge-10 left east -6 
            2  left east -7 
    
* CEM  most of wedge-6 east are noisy (RMS&gt;4) 
 Also QIE errors are larger for this particuler wedge 
  
*  PEM  QIE calibration: 
      wedge-22 west 11 is red in the slope of errors (for both runs) 
  
* Pedestal plug pin diode 
 wedge 2 left/right east (RMS&gt;2) 
  
*  WHA QIE large error 
   wedge 10 right east -10 
         19 right east -9 and -10 
         22 right west 10 
  
*  WHA noisy pedestal 
   wedge 11 left west 7 
         22 left west 6 
         20 lest east -7 
         23 left east -10 
         17 left east -8 
         13 right east -6 
         14 right east -6 
                                                                                                                             
  
 Arie Bodek (outgoing CO) Masa Tanaka (new CO day shift) 
 </pre>
<!-- Author: Masa, Arie -->


<!-- Date: Fri Jun 15 14:10:34 2001 -->
<pre>Date: Fri, 15 Jun 2001 14:08:06 -0500 (CDT) 
From: Masa Tanaka &lt;masa@hep.anl.gov&gt; 
To: byrum@fnal.gov 
Cc: bodek@fnal.gov 
Subject: CES 
 
  
CES wedge+we*24=46 is bad 
It is bad on pedestals and QIRE calibration. 
6/15/01 (Run 117825) also previous day calibration. 
  
Arie Bodek (outgoing CO), Masa (new CO) 
   </pre>
<!-- Author: Arie, Masa -->


<!-- Date: Fri Jun 15 21:43:17 2001 -->
<pre>TrigMon crashes every 15 minutes : 
FINAL FIT DONE! 
Display Server: End with signal 11 ( = SIGSEGV ) 
Report end to the StateManager 
 
 
XMon often crashes too. 
 
Gerry & Stephane, Eve shift</pre>
<!-- Author: tourneur@fnal.gov -->


<!-- Date: Sat Jun 16 16:07:57 2001 -->
<pre>LumMon has problem for reading histogram, 
likely caused by some junk in the data. 
It is fixed by the expert. 
 
Masa 
 
</pre>
<!-- Author: Masa -->


<!-- Date: Sat Jun 16 18:57:29 2001 -->
<pre>Finding it impossible to get TrigMon running during 
117925, a cosmic run....now that we have switched to 
beam triggers (117926) TrigMon starts up ok....but still 
hangs up after few min with fair regularity!!!!!!! 
</pre>
<!-- Author: gbauer -->


<!-- Date: Sun Jun 17 06:11:49 2001 -->
<pre>Kaori said that fix for TrigMon will be in next  
release of cdfsoftb0.  It crashes trying to close 
a file after ~1000 events.</pre>
<!-- Author: Amit Lath -->


<!-- Date: Mon Jun 18 11:47:18 2001 -->
<pre> cdfsoftb0 is updated (3.17.0b) this morning and now 
TrigMon is stable (at least ~2000 event) with 
the old 36x36 runs (partition 14).  
 
 In this version, another monitor (L3RegionalMon)  
is added, where, consumer is working on b0dap66 and display 
is working on top display of b0dap55 (with XMon).  
Type ./starMon L3RegionalMon &lt;pid&gt; and  
./startDisplay L3ReagionalMon for starting it. 
"All" option start all consumers including it. 
 
 There is a request from XMon to do at the end of the run 
by following the instructions in 
 /consumer/xmon/xmon_endrun.html . 
 
Masa 
 
 
 
 
</pre>
<!-- Author: Masa -->


<!-- Date: Tue Jun 19 01:06:49 2001 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Tue_Jun_19_2001_010627.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-30  align='top'></td><td>COT time dist for cosmic run.  Note L6 times look  different from the rest.  However COT HV was low/off (Aseet was downstairs monitoring things).  But still, L6 has lots more early hits than the rest...</a></td></tr></table>
<!-- Author: Amit  -->


<!-- Date: Tue Jun 19 07:18:09 2001 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Tue_Jun_19_2001_071755.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-32  align='top'></td><td>CMP Occupancy for Part S has hot channels.  I checked the same plot for run 117994 and these channels were not hot.</a></td></tr></table>
<!-- Author: Amit -->


<!-- Date: Tue Jun 19 07:47:55 2001 -->
<pre>Owl Shift CO Scorecard: 
 
Monitor             crashes 
---------------------------- 
TrigMon                6 
ObjectMon              3 
XMon                   3 
BeamMon                2 (display died 4 times) 
SiliMon                1 
LumMon                 1 
 
Hard earned wisdom: get the monitors running, and start displays only when you see lots of events going in the jobs. 
 
At run end TrigMon will surely die.  It may die after 
trigger inhibit as well.</pre>
<!-- Author: Amit -->


<!-- Date: Tue Jun 19 08:01:37 2001 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Tue_Jun_19_2001_080128.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-34  align='top'></td><td>CLTD has some chans with low efficiency.  These chans were low even for previous run 117884.</a></td></tr></table>
<!-- Author: Amit L, Rick T -->


<!-- Date: Tue Jun 19 14:56:24 2001 -->
<pre>   I made a form of Consumer crash list 
 file://cdf/people1/cdfdaq/consumer_scripts/check.html 
 Please print it out through netscape and use this sheet  
 for the crash log. 
 
</pre>
<!-- Author: masa -->


<!-- Date: Tue Jun 19 19:26:46 2001 -->
<pre>YMon consumer PC (b0dap52) had some processes sucking all the  
cpu such that even the cursor response was grinding to a halt. 
Killed offending process. After this i was unable to restart 
any display on 52...the monitoring process seemed to run ok. 
had following output: 
 
35 /cdf/people1/cdfdaq/consumer_scripts % ./startDisplay.sh LumMon 
 
This script should be executed by a Consumer Operator 
 
Now destroying any kerberos tickets which may exist 
 
kdestroy: No credentials cache file found while destroying cache 
Ticket cache NOT destroyed! 
/usr/bin/X11/xauth:  creating new authority file /tmp/Xauth5430_2144 
xhost: Command not found. 
access control disabled, clients can connect from any host 
/usr/bin/X11/xauth:  creating new authority file /tmp/Xauth5430_2206 
xhost: Command not found. 
Starting HistoDisplayMain on b0dap52 
36 /cdf/people1/cdfdaq/consumer_scripts % /usr/bin/X11/xauth:  creating new auth 
ority file /tmp/Xauth5430_2270 
xhost: Command not found. 
Xlib: connection to "b0dap52.fnal.gov:0.0" refused by server 
Xlib: Client is not authorized to connect to Server 
/usr/X11R6/bin/xterm Xt error: Can't open display: b0dap52.fnal.gov:0. 
 
finally stumbled on to fact that "xhost +" was sufficient 
to enable opening window. ("xhost +b0dap51" did not work). 
Don't really understand what happened. 
</pre>
<!-- Author: gbauer -->