<!-- Title: 11:24:56  Fri Aug 29 2003 -->
<!-- Crewchief: kaori -->
<!-- Op1: &nbsp; -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>new file, 4.11.0 and 4.11.2, etc.  RH^, 7 transition time.</pre>


<!-- Date: Fri Aug 29 11:27:15 2003 -->
<pre>b0dap50&gt; ups list -aK+ cdfsoftb0 
"cdfsoftb0" "4.10.0int3a" "Linux+2" "" ""  
"cdfsoftb0" "4.11.0a" "Linux+2" "" ""  
"cdfsoftb0" "4.11.2d" "Linux+2" "" ""  
"cdfsoftb0" "4.11.2e" "Linux+2" "" ""  
"cdfsoftb0" "4.10.0int1a" "Linux+2" "" ""  
"cdfsoftb0" "4.11.2g" "Linux+2" "" ""  
"cdfsoftb0" "4.11.0e" "Linux+2" "" "current"  
"cdfsoftb0" "4.11.0f" "Linux+2" "" ""  
 
4.11.2xxx are built under RH7.3.1 
and only runs on RH7 machine. 
</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 11:44:33 2003 -->
<pre>start_remote_4110f $partition b0dap77 XMon +400+250 
start_remote $partition b0dap76 YMon +400+0 
start_remote $partition b0dap65 SiliMon +400+500 
start_remote $partition b0dap77 LumMon +0+0 
start_remote_TrigMon_4112e $partition b0dap76 TrigMon +0+250 
start_remote $partition b0dap67 Stage0 +0+500 
start_remote_4112d $partition b0dap77 SVXMon +800+250 
start_remote_4110f $partition b0dap56 ObjectMon +0+750 
start_remote_4110f $partition b0dap56 ObjectMon +0+750 
start_remote $partition b0dap65 BeamMon +400+750 
start_remote $partition b0dap67 DAQMon +800+500 
</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 11:56:18 2003 -->
<pre>For EventDisplay, running on b0dap50, under cdfrun account, 
'startEvd.sh' will start 4.11.2g version on 'mon' which 
gets 'any triggered events', and handle the Q banks</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 14:45:08 2003 -->
<pre>Examine which monitors have runarea/xxxMon/Consumer/xxxMon/runxxxMon.tcl at a moment. 
 
BeamMon 
SiliMon 
TrigMon 
YMon 
ObjectMon 
Stage0 
 
--------- 
The monitors do not have overwriting tcl are: 
DAQMon, LumMon, XMon 
---------- 
 
SVXMon is special. 
SVXMon has tcl in: 
~igv/SvxMonConfig/ 
 
That area always exits.  If there is a file: 
"runSVXMon.tcl" in that area,  then SVXMon will use 
the tcl file.  If not, it uses default release tcl. 
At a moment, there is no runSVXMon.tcl in that area. 
 
When we run 4.11.2xxx release, we use the release tcl. 
When we run older SVXMon (4.11.0a, for example), we need 
to use runSVXMon.tcl_streamAD_kaori.  so do: 
cp runSVXMon.tcl_streamAD_kaori runSVXMon.tcl 
in ~igv/SvxMonConfig/  area using cdfdaq account.</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 16:14:51 2003 -->
<pre>Change tcls from whatevent there are to 4.11.2g version: 
 
for: 
YMon</pre>
<!-- Author: kaori -->
<!-- Comment: Fri Aug 29 16;29;02 comment by...kaori -->
<pre>run 4.11.2g version of YMon on b0dap76(partition 14), then YMon
runs ok.

run 4.11.0a version on b0dap76 (RH7), then it crashes.
run 4.11.0a version on b0dap51 (RH6), then it runs ok.

Keep 4.11.2g version YMon running on b0dap76.</pre>
<!-- Date: Fri Aug 29 16:26:47 2003 -->
<pre>Keep followings: 
b0dap77		XMon, LumMon, SVXMon 
b0dap76 	YMon, TrigMon 
 
Move followings: 
b0dap65 -&gt; b0dap72  SiliMon, BeamMon 
b0dap67 -&gt; b0dap57  Stage0, DAQMon 
b0dap56 -&gt; b0dap53  ObjectMon</pre>
<!-- Author: kaori -->
<!-- Comment: Fri Aug 29 16;30;15 comment by...kaori -->
<pre>Before chaning startMon.sh, save as startMon.sh_keep
</pre>


<!-- Date: Fri Aug 29 16:50:54 2003 -->
<pre>BeamMon tcl in /data1/consumer/runarea is different from the 
one in the cvs head. (4.11.2g version).  I don't know which one 
to use.  Keep what in there (/data1) for now, and meanwhile, sent 
mail to Hartmut.</pre>
<!-- Author: kaori -->
<!-- Comment: Fri Aug 29 17;07;40 comment by...kaori -->
<pre>Stage0 tcl on runarea/Consumer... was created by 'burkett'.
Called up Kevin and he removed the directory.  The old tcl file
is saved in runarea/Stage0/tcl_archive/runStage0.tcl_4110e.
</pre>

<!-- Comment: Fri Aug 29 17;14;37 comment by...kaori -->
<pre>copy runSiliMonPHYSICS.tcl to runSiliMonPHYSICS.tcl_4110e
copy runSiliMonSTANDBY.tcl to runSiliMonSTANDBY.tcl_4110e,

then copy 4.11.2g release ones to runarea/SiliMon/Consumer/SiliMon
area.</pre>

<!-- Comment: Fri Aug 29 17;31;56 comment by...kaori -->
<pre>Assume TrigMon tcl is upto date.  Copy runTrigMon.tcl_4112e
to tcl_archive area under runarea/TrigMon. then remove
runarea/TrigMon/Consumer directry.</pre>


<!-- Date: Fri Aug 29 18:00:33 2003 -->
<pre>ObjectMon tcl's (cosmic and physics) are copied as tcl_4110f, 
and copy cosmic and physics ObjectMon tcls from 4.11.2g area to 
runarea.</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 21:19:55 2003 -->
<pre>Well, there are some consumers happy running with partition 14 
now.  Happy ones are: 
BeamMon 
LumMon 
SVXMon 
SiliMon 
XMon 
YMon 
 
Unhappy ones are: 
TrigMon 
Stage0 
ObjectMon and DAQMon. 
--- 
Stage0 and DAQMon, I am trying to run on b0dap57, and ObjectMon 
I am trying to run on b0dap53.  Problem is trying to use cdfdaq 
account.  will change to 'cdfrun' account. 
I don't know why TrigMon crashes, yet.</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 21:52:27 2003 -->
<pre>ObjectMon (4.11.2g) fails in exe tcl stage. 
Put back 4.11.0e. back on b0dap56. then ok.</pre>
<!-- Author: kaori -->
<!-- Comment: Fri Aug 29 22;11;41 comment by...kaori -->
<pre>Was running 4.11.0f before.  So, I put back 4.11.0f for
objectMon.
</pre>
<!-- Date: Fri Aug 29 22:10:39 2003 -->
<pre>So, manage to run as much monitors as possible with 4.11.2g. 
 
BeamMon b0dap72.fnal.gov 
DAQMon	b0dap76.fnal.gov 
LumMon	b0dap77.fnal.gov 
SiliMon	b0dap72.fnal.gov 
Stage0_	b0dap76.fnal.gov 
XMon	b0dap77.fnal.gov	 
YMon	b0dap76.fnal.gov 
ObjectMon	b0dap56   4110f 
SVXMon  b0dap77 	  Igor special 
TrigMon b0cap76           4112e</pre>
<!-- Author: kaori -->
<!-- Comment: Fri Aug 29 22;32;17 comment by...kaori -->
<pre>So, ObjectMon is the only one running with exe built under
RH6.  all the others are running with 4.11.2 based exes.

Problem now is that we do not have enough RH7 machine to run
comfortably.  b0dap57 is RH7, but I am not sure I can use this
as cdfdaq account.

So, tonight, I'll go back to run RH6 built exes for some of
the monitors (which has no need to move around tcl's), in order
to spread out over machines.

Here are New config:
(need to move machines, and releases, but no need to
change tcl when we go back to 4.11.2g)

Only have to do this because not enough RH7 machine 
(with cdfdaq account to run all these monitors).

SiliMon b0dap72.fnal.gov  4.11.2g
TrigMon b0cap72           4.11.2e (move to 72)
Stage0_ b0dap76.fnal.gov  4.11.2g
YMon    b0dap76.fnal.gov  4.11.2g

BeamMon =&gt; b0dap65  run with 4.11.0e
XMon    =&gt; b0dap66  run with 4.11.0e
DAQMon  =&gt; b0dap67  run with 4.11.0e
LumMon  =&gt; b0dap67  run with 4.11.0e

SVXMon  b0dap77           Igor special 
TrigMon b0cap77           4112e (move to 77)

ObjectMon       b0dap56   4110f 
=================</pre>


<!-- Date: Fri Aug 29 23:39:06 2003 -->
<pre>Oops,  YMon crashed. 
Send e-mail to Adam (cc to wgw). 
Put YMon back running on 4.11.0e. 
but on b0dap76, still crashes (differently). 
run on b0dap65 with 4.11.0e.  Then ok. 
Leave on 65 with 4.11.0e. 
</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 23:40:35 2003 -->
<pre>This time, SVXMon crashes: 
 
sendString = SVXMon|SVXMon$SVXMonModule$Silicon/Softwar 
e/SVXMonModule::event$SvxMon:HeartBeat$11$168489/21773$ 
run = 168489 event = 21773$run = 168489 event = 21773$S 
VXMon is alive $::nEvents:: $0$::nStat:: $2$:: $ 
$END 
sendString = SVXMon|SVXMon$SVXMonModule$Silicon/Softwar 
e/SVXMonModule::event$SvxMon:HeartBeat$11$168489/21773$ 
run = 168489 event = 21773$run = 168489 event = 21773$S 
VXMon is alive $::nEvents:: $0$::nStat:: $2$:: $ 
$END 
sendString = SVXMon|SVXMon$SVXMonModule$Silicon/Softwar 
e/SVXMonModule::event$SvxMon:HeartBeat$11$168489/21773$ 
run = 168489 event = 21773$run = 168489 event = 21773$S 
VXMon is alive $::nEvents:: $0$::nStat:: $2$:: $ 
$END 
DisplayServer: Got an unexpected string from the consum 
er. Expected: 'CONSUMER SEND'. Received: '---'. 
DisplayServer::loop(): ERROR: poll consumer returned wi 
th: -5 
DisplayServer::loop(): Protocol between consumer and di 
splay server 
was not obeyed! Display Sever will exit! 
runSVXMon: Consumer ended with value  0. 
Press C-d to close window. 
 
========= 
I'll just send mail to Igor, Henri (cc to wgw) 
 
</pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 23:49:15 2003 -->
<pre>Current Summary, again: 
 
SiliMon	     b0dap72   4.11.2g 
TrigMon	     b0cap72   4.11.2e  
Stage0_	     b0dap76   4.11.2g 
YMon	     b0dap65   4.11.0e 
 
BeamMon	     b0dap65  run with 4.11.0e 
XMon	     b0dap66  run with 4.11.0e 
DAQMon	     b0dap67  run with 4.11.0e 
LumMon	     b0dap67  run with 4.11.0e 
 
SVXMon	     b0dap77   Igor special  
TrigMon	     b0cap77   4112e  
 
ObjectMon    b0dap56   4110f </pre>
<!-- Author: kaori -->


<!-- Date: Fri Aug 29 23:50:46 2003 -->
<pre>EventDisplay is running on b0dap50, 4.11.2g version. 
 
HistoDisplayMain is running with 4.11.0e, except for 
SVXMon Display is running with 4.10.0int1a on the lower 
b0dap56.</pre>
<!-- Author: kaori -->


<!-- Date: Sat Aug 30 08:01:17 2003 -->
<pre>Problem with HistoDisplay displaying YMon on RH7 b0dap54. 
After double clicking several canvases, the display loose  
socket connection and freezes.  Only way to get started is by 
killing display and restart.  This happens too often to be 
useable.  Version running was 4.11.0e.  I tried running 4.11.2g 
version on b0dap54, but that did the same bad.  Went back to 
b0dap55 (RH6) tried 4.11.0e.  Some reason 'choose' window does 
not pop up, and freeze after clicking 'choose'.  Go back even 
farther back, using 4.10.0int1a (same as what we are using for 
SVXMon display), then it is ok.  So, leave YMon and TrigMon 
display running on b0dap55, YMon display with 4.10.0int1a, and 
TrigMon display with 4.11.0e.   </pre>
<!-- Author: kaori -->


<!-- Date: Sun Aug 31 09:05:01 2003 -->
<pre>SVXMon display is going 'red'. 
HistoDisplayMain version running is 4.10.0int1a.  We have never 
seen this problem before with this HistoDisplay version. 
SVXMon version running is igor's special version. 
 
For the next run, as a test, I run my release version of 
SVXMon, 4.11.2d (which showed no problem with 'red' with  
4.10.0int1a version of display). 
 
To run my version, I'll do: 
 
rm ~igv/svxmon_current/bin/Linux2-KCC_4_0/SVXMon 
 
After, to install back igor's version, I should do: 
 
cp ~igv/svxmon_current/bin/Linux2-KCC_4_0/SVXMon.good 
~igv/svxmon_current/bin/Linux2-KCC_4_0/SVXMon 
chmod a+w ~igv/svxmon_current/bin/Linux2-KCC_4_0/SVXMon 
 
In any case, we do not have to have a special tcl. 
tcl place is ~igv/SvxMonConfig/  and neither igor special, 
nor 4.11.2d version require special runSVXMon.tcl in the 
are.  (I am checking the area, and I don't see runSVXMon.tcl 
right now.  Right now, it is still how Henri set on Friday 
for igor special exe.)</pre>
<!-- Author: kaori -->
<!-- Comment: Sun Aug 31 09;17;06 comment by...kaori -->
<pre>I did change SVXMon to my release, and restarted.
run # going on is 168561</pre>
<!-- Comment: Sun Aug 31 10;11;42 comment by...kaori -->
<pre>Previous try of setting SVXMon version to 4.11.2d did not fix
the 'red' problem.

Now, change SVXMon to even older version, 4.10.0int3a which is 
the older ROOT version running for a long time without 'red'
problem.   tcl file now have to explicitely made for this.
I put the explicit tcl.  With 4.10.0int3a SVXMon and 4.10.01a
Display, 'red' problem is gone.  In any case, this comibination
is what i have been running with, for a quite long time without
the 'red' problem.  If the new SVxMon version will be put back
in, then someone need to make sure to remove the tcl. 
With David Clark discussion, we are leaving this good old combo.
as a default for now.</pre>
<!-- Date: Sun Aug 31 09:06:29 2003 -->
<pre>XMon is set to run with 4.11.0f (since Friday, it was set to 
run with 4.11.0e).  this caused problem with XMon web page in 
Charles private area.  It should be ok, now.  Restarted XMon. 
</pre>
<!-- Author: kaori -->




<!-- Date: Wed Sep 3 17:38:17 2003 -->
<pre>SVX people want to make a test with 2 SRCs. 
This requires the newest SVXMon version available. 
As far as I can see we have no release version of SVXMon that 
has the needed options. I looked at 4.11.2g, the newest release 
built on Aug. 28 together with Jason Nielson. We came to the conclusion that this code does not have the stuff to run with 2 SRC. 
 
We need to run a special version built by Henri on Friday. 
We copy  
/data1/igv/svxmon_4.11.1/bin/Linux2-KCC_4_0/SVXMon 
to Igors runarea: 
/cdf/people1/igv/svxmon_current/bin/Linux2-KCC_4_0 
 
</pre>
<!-- Author: Wolfgang -->


<!-- Date: Wed Sep 3 21:42:11 2003 -->
<pre>We are now trying to make new release 4.11.2h on b0dap34(RH7) with new SVXMon and HistoDisplayMain to resolve "red problem". 
Now it is under compiling, so once we have done this successfully, we will try to test it.</pre>
<!-- Author: Kohei and Wolfgang -->


<!-- Date: Wed Sep 3 23:18:13 2003 -->
<pre>Now compile and building binary have been done successfully(means no crach error). 
When I made this version, I deleted the file of SvxDaqUtils/src/ReadSiChipOnAsciiFiles.cc by following Igor's instruction. 
 
We will test it tomorrow morning.  
 
</pre>
<!-- Author: Kohei -->


<!-- Date: Thu Sep 4 11:42:06 2003 -->
<pre>While test new SVXMon and HistoDisplayMain with new release 4.11.2h, some "red problem" have been occured. 
See Silicon e-log in more details. 
 
We are still checking/investigating SVXMon problem right now. </pre>
<!-- Author: kohei -->


<!-- Date: Fri Sep 5 18:38:52 2003 -->
<pre>4.11.0a was deleted from cdfsoftb0 release since disc space in that area is almost full(95%). after Art removed some files, it is now 84%.</pre>
<!-- Author: kohei -->


<!-- Date: Tue Sep 9 16:35:58 2003 -->
<pre>Now I am trying to make new release 4.11.2i that contains fixed HistoDisplayMain(no freeze problem). 
 </pre>
<!-- Author: kohei -->


<!-- Date: Tue Sep 9 19:51:42 2003 -->
<pre>4.11.2i was built successfully(no crashing)</pre>
<!-- Author: kohei -->


<!-- Date: Wed Sep 10 11:51:22 2003 -->
<pre>declared 4.11.2i as current default cdfsoftb0 version.  Previous 
default was 4.11.0e. 
 
startDisplay.sh and killAllDisplays.sh are working. 
 
Now, we are working on startMon.sh with the new 'current' version. 
 
Monitor testing is being done using partition 14 at a moment. 
Monitors taking some events, (running ok) are: 
BeamMon 
DAQMon 
LumMon 
SiliMon 
Stage0 
 
YMon ---  had problem with tcl, but was an old one.  copied new 
          tcl files from release, then runs ok. 
 
ObjectMon --- had problem with tcl.  copied new one.  still 
              problem with "slideshowAllPt".  sent mail to Vadim. 
              Modify tcl in runarea, comment out "slideshowAllPt" 
              in SVXMonitor part. 
====================== 
Stage0, YMon, SiliMon, etc. crashed at the end of run. 
ErrorReceiver, StateManagers are still running on RH6 machine. 
 
 
 
 
 
</pre>
<!-- Author: kaori -->


<!-- Date: Wed Sep 10 12:55:32 2003 -->
<pre>TrigMon does not run with the new cdfsoftb0 release, 4.11.2i. 
 
gdb output shows: 
Reading symbols from /lib/libnss_nis.so.2...done. 
Loaded symbols for /lib/libnss_nis.so.2 
#0  0x7473696c in ?? () 
(gdb) where 
#0  0x7473696c in ?? () 
#1  0x86d57f9 in ConsumerFrameworkModule::beginJob (this=0xae4f748, __T156555632=0xbfff9344,  
    __290825_55_anEvent=0xaeced38) 
    at /mnt/code-linux/cdfoffline/dist/cdfsoftb0/4.11.2i/ConsumerFramework/src/ConsumerFrameworkModule.cc:132 
#2  0x890b0bf in AppFramework::beginJob (this=0xade7620, __T152771820=0xbfff93d4,  
    __195131_35_anEvent=0xaeced38) 
    at /home/cdfsoft/dist/packages/Framework/V00-07-32/src/APPFramework.cc:319 
#3  0x890dbff in AppFramework::beginHandler (this=0xade7620, __195639_33_nEvents=2147483647) 
    at /home/cdfsoft/dist/packages/Framework/V00-07-32/src/APPFramework.cc:811 
#4  0x893e354 in APPEventsCommand::beginHandler (this=0xade77ec, __161188_37_argc=4,  
    __161188_49_argv=0xbfff94c8) 
    at /home/cdfsoft/dist/packages/Framework/V00-07-32/src/APPEventsCommand.cc:145 
 
====== 
not much help here. 
We tried running gdb with TrigMon running the same time. 
Information is the same as above. 
</pre>
<!-- Author: kaori -->
<!-- Comment: Wed Sep 10 12;59;42 comment by...kaori -->
<pre>In the startup script "startMon.sh", all the monitor but TrigMon
are set to startup with new default cdfsoftb0 version 4.11.2i.
TrigMon is set to startup with 4.11.2e.
TrigMon runarea does not have Consumer/TrigMon area.  So
the tcl is not an issue in this end.</pre>


<!-- Date: Wed Sep 10 13:08:29 2003 -->
<pre>moved:  
mv runSVXMon.tcl runSVXMon.tcl_91003_kaori 
 
in ~igv/SvxMonConfig/ 
 
and trying SVXMon using partition 14, again.</pre>
<!-- Author: kaori -->
<!-- Comment: Wed Sep 10 13;13;16 comment by...kaori -->
<pre>SVXMon now processing events.</pre>


<!-- Date: Sat Sep 13 08:54:57 2003 -->
<pre>We had a problem that ErrorReceiver was not built on 
b0dap34. 
following is a copy of mail from wolfgang and Art how 
to fix the problem: 
 
On Fri, 12 Sep 2003, Wolfgang Wagner wrote: 
... 
&gt; I guess somewhere there is some statement like 
&gt; if hostname = b0dap32 then setenv CDF_ONLINE_CONSUMERS TRUE. 
&gt; And b0dap34 would have to be added to this 
 
This is done in ~cdfsoft/cdf_local.[c]sh . 
 
I have just copied these files to b0dap34 from b0dap32 .</pre>
<!-- Author: kaori -->


<!-- Date: Sat Sep 13 09:02:09 2003 -->
<pre>Making a new release, 4.11.2j. 
 
This should have: 
1) ErrorReceiver exe. 
2) EventDisplay has an improvement by Andrei that 
   we can display absolute scale..  (not sure this is 
   actually committed, yet..) 
 
Patch file looks: 
#Consumer patches 
Consumer -D 13-Sep2003UT1300 
ConsumerFramework -D 13-Sep2003UT1300 
 
#packages needed for TrigMon and XMon 
XTRPSim -D 13-Sep2003UT1300 
CalTrigger -D 13-Sep2003UT1300 
L1GlobalTrigger -D 13-Sep2003UT1300 
XFTSim -D 13-Sep2003UT1300 
 
#package for SVXMon 
SvxDaqMods -D 13-Sep2003UT1300 
SvxDaqUtils -D 13-Sep2003UT1300 
 
#package for EventDisplay 
RootMods -D 13-Sep2003UT1300 
RootEventDisplay -D 13-Sep2003UT1300 
 
#packages for the compression of COT and Silicon banks  
#PADSMods -r v4_8_5b 
#PADSObjects -r l3_4_8_4_q3 
PADSMods  -D 13-Sep2003UT1300 
PADSObjects -D 13-Sep2003UT1300 
 
#this to be the last line of the patch file 
================= 
right after addpkg, remove: 
SvxDaqUtils/src/ReadSiChipOnAsciiFiles.cc 
before building (as kohei did to make 4.11.2i)</pre>
<!-- Author: kaori -->


<!-- Date: Sat Sep 13 09:05:09 2003 -->
<pre>Trying to use b0dap65,66 and 67. 
Modified /cdf/people1/cdfdaq/.ssh/known_hosts on b0dap51 to 
stop asking me yes/no quesiont.  But it did not work. 
 
So, startMon.sh how I want is now copied to startMon.sh_new, 
and startMon.sh (at a moment) is still not using 65,66, and 67.</pre>
<!-- Author: kaori -->


<!-- Date: Mon Sep 15 10:10:13 2003 -->
<pre>starting XMon partition 5 on b0dap66 
using 65, 66, 67, 76, and 77 now 
using startMon.sh 
 
starting YMon partition 5 on b0dap65 
starting SiliMon partition 5 on b0dap66 
starting LumMon partition 5 on b0dap77 
starting TrigMon partition 5 on b0dap77 
starting Stage0 partition 5 on b0dap67 
starting SVXMon partition 5 on b0dap76 
starting ObjectMon partition 5 on b0dap76 
starting BeamMon partition 5 on b0dap65 
starting DAQMon partition 5 on b0dap67 
</pre>
<!-- Author: kaori -->


<!-- Date: Mon Sep 15 10:11:18 2003 -->
<pre>Decleared 4.11.2j as current default.   
4.11.2j is pulled to b0dap32.  "pull_cdfsoftb0 4.11.2j b0dap34" 
This release has ConsumerErrorReceiver.</pre>
<!-- Author: kaori -->


<!-- Date: Fri Sep 19 18:59:41 2003 -->
<pre>I post here an e-mail sent to TrigMon people about 
the recently seen TrigMon crashes for documentary purposes. 
 
Date: Sat, 20 Sep 2003 01:56:54 +0200 (CEST) 
From: Wolfgang Wagner &lt;wagner@ekp.physik.uni-karlsruhe.de&gt; 
X-X-Sender: wagner@ekplx22.physik.uni-karlsruhe.de 
To: Charles Plager &lt;cplager+ucla@fnal.gov&gt;, "" &lt;maeshima@fnal.gov&gt;, 
    "" &lt;kohei@fnal.gov&gt;, "" &lt;ngoldsch@umich.edu&gt;, "" &lt;rolli@fnal.gov&gt;, 
    "" &lt;jameseb@fnal.gov&gt;, "" &lt;jiank@fnal.gov&gt;, "" &lt;mworcest@fnal.gov&gt; 
Subject: TrigMon crash solved 
In-Reply-To: &lt;3F6B5E77.6020609@fnal.gov&gt; 
Message-ID: &lt;Pine.LNX.4.50.0309200057310.5996-100000@ekplx22.physik.uni-karlsruhe.de&gt; 
References: &lt;3F6B5E77.6020609@fnal.gov&gt; 
MIME-Version: 1.0 
Content-Type: TEXT/PLAIN; charset=US-ASCII 
 
Hi all, 
 
the TrigMon crash problem is solved. The issue is the following: 
 
L2TriggerMonitor and MuonTriggerMonitor are not part of the 
Consumer package. If ConsumerFramework, BaseMonitor2 specifically, 
changes, the corresponding packagees MuonTrigger and L2GlobalTrigger 
have to be added to the test release when TrigMon is built. 
 
If that is not done MuonTriggerMonitor and L2TriggerMonitor inherit 
from the old base release version of BaseMonitor2, but when linking 
the TrigMon executable, the new ConsumerFramework is linked. 
In certain cases this might be figured out by the linker and you 
get error messages. But it does not necessarily have to be that way, 
as in our current case. 
 
So as a golden rule for building test releases or cdfsoftb0 
releases it has to be said, that 
"Whenever ConsumerFramework changes, you have to add all packages 
to your test release (or cdfsoftb0) which depend on ConsumerFramework." 
 
Most online consumer code which depends on ConsumerFramework are 
contained in the "Consumer" package. This makes life easy because 
you don't need to account for additional dependencies. 
 
TrigMon, with L2TriggerMonitor and MuonTriggerMonitor, are different, 
because they reside in different packages. They need to be checked out 
in addition. Which means additional work and resources when building 
the online cdfsoftb0 releases. 
 
I think it would alleviate the task of building a cdfsoftb0 release 
quite a bit, if these criss-cross dependencies were resolved. 
I therefore suggest to the trigmon group and the monitor authors 
to move L2TriggerMonitor and MuonTriggerMonitor to the Consumer package. 
 
I'm afraid if no action of that kind is taken, we will stumble over that 
trap again and again, thereby consuming a lot of valuable time of people 
like Kaori, Charles and myself. 
 
Best regards, 
                 Wolfgang 
 
------------------------------------------------------------------------- 
 Dr. Wolfgang Wagner 
 Universitaet Karlsruhe - Institut fuer Experimentelle Kernphysik (EKP) 
 Wolfgang-Gaede-Strasse 1, 76128 Karlsruhe 
 
 E-Mail: wagner@ekp.physik.uni-karlsruhe.de        Tel.: +49-721-608-4706 
------------------------------------------------------------------------- 
</pre>
<!-- Author: Wolfgang -->


<!-- Date: Mon Sep 22 09:02:23 2003 -->
<pre>We need to build a new release to fix the TrigMon problem. 
We will include 3 packages to the patches in addition to those 
which we already have: 
XFTMods -D 04-Sep2003UT0140 
MuonTrigger -D 04-Sep2003UT0140 
L2GlobalTrigger -D 04-Sep2003UT0140 
 
This is in patches file: 4.11.2k.patches 
Kohei will start building the new release.</pre>
<!-- Author: Wolfgang / Kohei -->


<!-- Date: Mon Sep 22 10:01:18 2003 -->
<pre>Now new release 4.11.2k is built on b0dap32. 
</pre>
<!-- Author: kohei -->


<!-- Date: Mon Sep 22 10:06:24 2003 -->
<pre>YMon had a problem of starting with the startup script. 
Fixed startSubConsumer.csh script (commmented out old rcp 
mods which should be commented out).  Now YMon starts up ok.</pre>
<!-- Author: kaori -->


<!-- Date: Mon Sep 22 12:41:24 2003 -->
<pre>4.11.2k was built with no crash on b0dap32 and declared to UPS.</pre>
<!-- Author: kohei -->


<!-- Date: Mon Sep 22 16:32:27 2003 -->
<pre>Made 4.11.2k as a default current cdfsoftb0 and tested with 
cosmic run in the control room.  Now TrigMon runs without crashing.  All the other monitors, HistoDisplay, EventDisplay 
runs without crashing. 
So, I leave 4.11.2k as current default. 
 
There are a few problems still need to be  looked into: 
1) archive job on b0dap32 (runs under cdf_cf, cron job supposed 
to take care of it)has not been running since b0dap32 update. 
Started by hand. 
 
2) SVXMon starts spewing a huge amount of error messages: 
 
l3_csl_socket_plumber: SIGPIPE signal... broken connection 
sendString = SVXMon|SVXMon$SVXMonModule$Silicon/Software/SVXMonModule::event$SvxMon:HeartBeat$11$169567/70463$run = 169567 event = 70463$run = 169567 event = 70463$SVXMon is alive $::nEvents:: $0$::nStat:: $3$:: $ 
$END 
 
l3_csl_socket_plumber: SIGPIPE signal... broken connection 
sendString = SVXMon|SVXMon$SVXMonModule$Silicon/Software/SVXMonModule::event$SvxMon:HeartBeat$11$169567/70463$run = 169567 event = 70463$run = 169567 event = 70463$SVXMon is alive $::nEvents:: $0$::nStat:: $3$:: $ 
$END 
 
-------- 
after while.  Too fast to kill it by hitting ctrl_C. 
 
Don't know why this happens.  Set SVXMon back to 4.11.2j 
(previous version).  Perhaps the file supposed to remove before 
gmake was not done? 
 
3) YMon crashes after some time (approx. 10 min from start) 
sendString = YMon|26080@b0dap65.fnal.gov$YMonModule$CMX 
$CMX Monitor$7$169567/99008$run = 169567 event = 99008$ 
run = 169567 event = 99008$Geometric indices out of ran 
ge in CMXD $ ierror: $1$ side: $1$ wedge: $6$ wire: $2$ 
 layer: $2$ 
$END 
%ERLOG-e CMX Monitor: Geometric indices out of range in 
 CMXD  ierror: 1 side: 1 
          wedge: 6 wire: 2 layer: 2 YMonModule CMX() 22 
-Sep-2003 16:29:16  
         run = 169567 event = 99008 
sendString = YMon|26080@b0dap65.fnal.gov$YMonModule$CMX 
$CMX Monitor$7$169567/99008$run = 169567 event = 99008$ 
run = 169567 event = 99008$Geometric indices out of ran 
ge in CMXD $ ierror: $1$ side: $1$ wedge: $6$ wire: $2$ 
 layer: $2$ 
$END 
ConsumerErrorSender::send(): SIGPIPE flag was raised. T 
he socket to 
the ConsumerErrorReceiver is potentially broken. We clo 
se the socket and delete the TSocket object. 
%ERLOG-s ROOT/TUnixSystem: send (Broken pipe)  GlobalLi 
braryLogger  
         TUnixSystem::UnixSend() 22-Sep-2003 16:29:16 r 
 
========== 
 
========== 
===== 
So, I set YMon to run with 4.11.2j back, also. 
 
</pre>
<!-- Author: kaori -->


<!-- Date: Mon Sep 22 16:37:29 2003 -->
<pre>The behaviour of SVXMon seems to be same for 4.11.2j 
and 4.11.2k.  So the problem is elsewhere.</pre>
<!-- Author: kaori -->


<!-- Date: Wed Sep 24 08:43:12 2003 -->
<pre>Let's delete some old cdfsoftb0 releases. 
At a moment, we have: 
 
[cdfsoft@b0dap32 ~]$ ups list -aK+ cdfsoftb0 
"cdfsoftb0" "4.10.0int3a" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2i" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2d" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2e" "Linux+2" "" "" 
"cdfsoftb0" "4.10.0int1a" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2g" "Linux+2" "" "" 
"cdfsoftb0" "4.11.0e" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2h" "Linux+2" "" "" 
"cdfsoftb0" "4.11.0f" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2j" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2k" "Linux+2" "" "current" 
 
============ 
cleaned up most of them. 
[cdfsoft@b0dap32 cdfsoftb0]$ ups list -aK+ cdfsoftb0 
"cdfsoftb0" "4.11.2j" "Linux+2" "" "" 
"cdfsoftb0" "4.11.2k" "Linux+2" "" "current" 
 
 
/cdf/code-Linux disk space usage went down from 91% to 64%. 
 
 
Now we have: 
 
 
======= 
</pre>
<!-- Author: kohei/kaori -->


<!-- Date: Wed Sep 24 08:47:41 2003 -->
<pre>At a moment, all the consumer monitor related processes are  
running under 'current' cdfsoftb0 release, 4.11.2k. 
Tested with cosmic run (partition 7) in real time, and all 
are running ok.  SVXMon is just sitting there not processing 
events as it should.</pre>
<!-- Author: kohei/kaori -->


<!-- Date: Thu Sep 25 08:56:03 2003 -->
<pre>We are trying to make new release 5.1.0a on b0dap32. 
to do this, 5.1.0a.patches is, 
========= 
#Consumer patches 
Consumer -D 25-Sep2003UT1330 
ConsumerFramework -D 25-Sep2003UT1330 
 
#packages needed for TrigMon and XMon 
XTRPSim -D 25-Sep2003UT1330 
CalTrigger -D 25-Sep2003UT1330 
L1GlobalTrigger -D 25-Sep2003UT1330 
XFTSim -D 25-Sep2003UT1330 
XFTMods -D 25-Sep2003UT1330  
MuonTrigger -D 25-Sep2003UT1330  
#L2GlobalTrigger -D 25-Sep2003UT1330  
 
#package for SVXMon 
SvxDaqMods -D 25-Sep2003UT1330 
SvxDaqUtils -D 25-Sep2003UT1330 
 
#package for EventDisplay 
#RootMods -D 25-Sep2003UT1330 
#RootEventDisplay -D 25-Sep2003UT1330 
 
#this to be the last line of the patch file 
========= 
 
Changes from previous release 4.11.2k are 
remove  
1) PADMods,PADSObjects(for compression) 
2) RootMods,RootEventDisplay 
3) L2GlobalTrigger(already in Consumer package) 
</pre>
<!-- Author: kaori/kohei -->


<!-- Date: Thu Sep 25 19:21:01 2003 -->
<pre>Commited the following patch file CVS. 
4.10.0int1a.b0.patches 
4.10.0int3a.b0.patches 
4.11.0e.b0.patches 
4.11.0f.b0.patches 
4.11.2d.b0.patches 
4.11.2e.b0.patches 
4.11.2g.b0.patches 
4.11.2h.b0.patches 
4.11.2i.b0.patches 
4.11.2j.b0.patches 
4.11.2k.b0.patches 
 
</pre>
<!-- Author: Kohei -->


<!-- Date: Thu Sep 25 19:24:49 2003 -->
<pre>While compiling Stage0,(COTDMonitor.cc), we have got the following message, 
 
======= 
"/cdf/code-Linux/cdfoffline/dist/cdfsoftb0/5.1.0a/Consumer/Stage0/COTDMonitor.cc", line 1834: error: 
          class "CT_DriftModelA" has no member "itrise" 
      if (_fixTRise.value())  _fit[sl].fix(CT_DriftModelA::itrise); 
=======                                                            
 
So Stage0 was not built in 5.1.0a.</pre>
<!-- Author: kohei -->


<!-- Date: Fri Sep 26 23:02:51 2003 -->
<pre>Again, try to make 5.1.0b with PadMods to make compression bank module compiled. patch file is, 
===== 
Consumer patches 
Consumer -D 27-Sep2003UT0330 
ConsumerFramework -D 27-Sep2003UT0330 
  
#packages needed for TrigMon and XMon 
XTRPSim -D 27-Sep2003UT0330 
CalTrigger -D 27-Sep2003UT0330 
L1GlobalTrigger -D 27-Sep2003UT0330 
XFTSim -D 27-Sep2003UT0330 
XFTMods -D 27-Sep2003UT0330 
MuonTrigger -D 27-Sep2003UT0330 
#L2GlobalTrigger -D 27-Sep2003UT0330 
  
  
#package for SVXMon 
SvxDaqMods -D 27-Sep2003UT0330 
SvxDaqUtils -D 27-Sep2003UT0330 
  
#packages for the compression of COT and Silicon banks 
#PADSMods -r v4_8_5b 
#PADSObjects -r l3_4_8_4_q3 
PADSMods  -D 27-Sep2003UT0330 
PADSObjects -D 27-Sep2003UT0330 
  
#package for EventDisplay 
#RootMods -D 27-Sep2003UT0330 
#RootEventDisplay -D 27-Sep2003UT0330 
  
#this to be the last line of the patch file 
=====</pre>
<!-- Author: kohei -->


<!-- Date: Sat Sep 27 18:23:37 2003 -->
<pre>I tried to make 5.1.0b with patch file in previous log, but got the same error and Stage0 was not built.</pre>
<!-- Author: kohei -->


<!-- Date: Wed Oct 1 09:15:46 2003 -->
<pre>YMon crashes, please find fragments of standard output below:  
 
It looks like it fails on the first event: 
 
APPConsumerInputModule::initIOSystem ===&gt; CS_ConsSend p 
ort = 4031 
APPConsumerInputModule::initIOSystem ===&gt; CS_Receiver p 
ort = 4030 
 Alignment:    on: 1    wafers on:1 
%ERLOG-s Abnormal termination: caused by signal:  8 Fra 
mework abortJob()  
          1-Oct-2003 08:58:18 run = 170006 event = 2633 
3 
sendString = YMon|14529@b0dap65.fnal.gov$Framework$abor 
tJob$Abnormal termination$11$170006/26333$run = 170006  
event = 26333$run = 170006 event = 26333$caused by sign 
al:  $8$ 
$END 
Calling abortJob for DummyInput... 
Calling abortJob for FileInput... 
 
=============================================================== 
This is output of the core file located in the runarea: 
 
#0  0x844d426 in FDLDMonitor::addEvent (this=0xa9e0078, __268970_41_record=0xb91d080) 
    at /cdf/code-Linux/cdfoffline/dist/cdfsoftb0/4.11.2k/Consumer/YMon/FDLDMonitor.cc:390 
#1  0x84e9428 in ConsumerFramework::addEvent (this=0xa9b5c40, __223116_47_record=0xb91d080) 
    at /cdf/code-Linux/cdfoffline/dist/cdfsoftb0/4.11.2k/ConsumerFramework/src/ConsumerFramework.cc:170 
#2  0x84eeb2a in ConsumerFrameworkModule::event (this=0xa9b5458, __T157020708=0xbfff9468, 
    __290902_52_anEvent=0xb91d080) 
    at /cdf/code-Linux/cdfoffline/dist/cdfsoftb0/4.11.2k/ConsumerFramework/src/ConsumerFrameworkModule.cc:202 
#3  0x86ab73e in APPSequence::event (this=0xa977ed0, __T149354448=0xbfff9504, 
    __160850_31_anEvent=0xb91d080) 
    at /home/cdfsoft/dist/packages/Framework/V00-07-32/src/APPSequence.cc:211 
#4  0x8698aff in AppFramework::event (this=0xa976c00, __T153548916=0xbfff95f8, 
    __195254_32_anEvent=0xb91d080) 
    at /home/cdfsoft/dist/packages/Framework/V00-07-32/src/APPFramework.cc:450 
#5  0x869b1bc in AppFramework::continueHandler (this=0xa976c00, 
    __195652_36_nEvents=2147483647) 
    at /home/cdfsoft/dist/packages/Framework/V00-07-32/src/APPFramework.cc:878 
 
</pre>
<!-- Author: Wolfgang -->


<!-- Date: Wed Oct 1 09:19:36 2003 -->
<pre>The line the crash happens in FDLDMonitor is 390: 
 
_PPbrCollVsBunch-&gt;Fill(f_bunch,(float)_ppbrCollBunch[i_bunch]*0.001); 
</pre>
<!-- Author: Wolfgang -->


<!-- Date: Mon Oct 6 09:10:46 2003 -->
<pre>Now trying to new release 5.1.0c with Stage0 COTDMonitor fix. 
after this(if success), we will test all monitor as soon as possible.</pre>
<!-- Author: kohei -->


<!-- Date: Mon Oct 6 13:44:32 2003 -->
<pre>5.1.0c was built successfully(with Stage0).</pre>
<!-- Author: kohei -->


<!-- Date: Thu Oct 9 14:42:15 2003 -->
<pre>List of things good to be fixed: 
 
1) YMon and XMon do not die completely when 
we hit ctl_C.  That is annoying.</pre>
<!-- Author: kaori -->
<!-- Comment: Thu Oct  9 14;59;31 comment by...kaori -->
<pre>2) make validate (end of the cdfsoftb0 release making) work</pre>

<!-- Comment: Thu Oct  9 15;15;55 comment by...kaori -->
<pre>3)  in general,  having .tcl files in /data1/consumer/runarea
is a real pain, when we try to test new releases.  It would be
a very good idea if there are better way of dealing with
/data1 tcl files.</pre>


<!-- Date: Thu Oct 9 15:48:58 2003 -->
<pre>We have decleared new cdfsoftb0 release 5.1.0c as  
'current default' release.  Tested with partition 14 and 
seems to be working ok.  ObjectMon crashes.  Works with 
turning off Cal module.  WGW is contacting responsible party.</pre>
<!-- Author: kaori -->


<!-- Date: Wed Oct 15 10:05:38 2003 -->
<pre>Consumers which use COT tracks produce lots of error 
messages because COTD bank is dropped and COTQ bank has to 
be uncompressed: 
 
The following code has to be added to the build job: 
  In your binary, add the header: 
  
#include "PADSMods/PadRawModule.hh" 
  
Then use the following lines to add the Cotq module before any modules that access the COT hits: 
  
module = new CotqModule(); 
add(module); 
 
thanks to Chris Hays for pointing that out. 
 
I will change BeamMon accordingly. 
</pre>
<!-- Author: Wolfgang -->


<!-- Date: Thu Oct 23 10:42:42 2003 -->
Move TrigMon and LumMon from b0dap77 to b0dap33 by Frank C's requests. ConsumerStageManger has not been properly functioning since Oct. 20th 11:45am, due to staled NSF connection (consequence is that Consumer HistoDisplay does not connect to the right consumers from the 'choose' window). This problem will be fixed by the DAQ people this afternoon after the DATABASE work is done and when b0dap32 gets rebooted.
<!-- Author: kaori -->