<!-- Title: 19:45:24  Mon Nov 26 2001 -->
<!-- Crewchief: Kaori -->
<!-- Op1: &nbsp; -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>Starting a new file, since previous one got so long.</pre>


<!-- Date: Mon Nov 26 19:50:26 2001 -->
<pre>Dear CO people: 
 
For now, please ignore L3RegionalMon.  James Loken who  
coded L3RegionalMon knows about the problem (to do with 
regional tracking reading tracking information....). 
 
In theory ( :=) ), event display should work now, by 
typing:  
./startEvd.sh &lt;partitionID&gt; 
 
Please try and if it does not work, let me know.</pre>
<!-- Author: kaori -->


<!-- Date: Mon Nov 26 20:37:17 2001 -->
<p>XMon stops execution in *.tcl file with the message:<p> 
invalid command name "debugLRIH" 
<p>
<!-- Author: Sasha -->


<!-- Date: Mon Nov 26 22:29:32 2001 -->
<p>TrigMon has crashed. When I was trying to restart it, it stopped in *.tcl file with the message:<p> 
invalid command name "CLAD"<p>
<!-- Author: Sasha -->


<!-- Date: Mon Nov 26 22:47:08 2001 -->
<p>Kaori called back and gave recomendation to use<p> 
./startMon_new.sh [Consumer] [PartID]<p> 
script to restart consumers. <p> Looks like it solves the problems with XMon and TrigMon. At least the consumers did not complaine on any invalid command and now they are waiting for events.<p>
<!-- Author: Sasha -->


<!-- Date: Tue Nov 27 01:38:25 2001 -->
<pre>XMon keeps crashing right at the beginning with many error messages including: 
 
===================================================== 
Retrieving L1/L2 triggers from database, run 133406.  D 
one. 
Getting L3 trigger paths and modules.  Done. 
Socket not valid. 
Socket not valid. 
Socket not valid. 
ConsumerInput Begin processing 100th record. Run 133406 
, 
====================================================== 
 
and further down: 
=========================================================== 
Error in &lt;TPad::RangeAxis&gt;: illegal axis coordinates ra 
nge: xmin=0.000000, ymin=-34601484.000000, xmax=50.0000 
00, ymax=-657428160.000000 
%ERLOG-s Abnormal termination: caused by signal:  8 Fra 
mework abortJob()  
         27-Nov-2001 01:29:18 run = 133406 event = 2936 
Calling abortJob for DummyInput... 
========================================================== 
 
I've given up on it.</pre>
<!-- Author: Jonas -->
<!-- Comment: Tue Nov 27 02;33;27 comment by...Jonas -->
I <em> am </em> using ./startMon_new.sh.

<!-- Comment: Tue Nov 27 19;58;20 comment by...Matt -->
This was the problem of the "negative luminosity" in XMon also seen by Kaori.  It is fixed for the next cdfsoftb0 release.  Until then, if the CLC is included in the run, but no luminosity is present, XMon will probably crash with this error.


<!-- Date: Wed Nov 28 04:48:46 2001 -->
<center><a href='/elog/notebooks/consumer-ops/Wed_Nov_28_2001_041121.ps'><img src='/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-138' ></a><a href='/elog/notebooks/consumer-ops/Wed_Nov_28_2001_041306.ps'><img src='/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-140' ></a><a href='/elog/notebooks/consumer-ops/Wed_Nov_28_2001_044604.ps'><img src='/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-142' ></a></center>
<!-- Author: Jonas -->
<!-- Comment: Wed Nov 28 04;57;42 comment by...Jonas -->
Two things are odd about the events seen in the event display:
<P>
1) Apart from the 3TeV tower that is always there, there is no activity in the calorimter seen on the event display. This can be explained by assuming that root scales all the other towers to accommodate the 3TeV tower, so even though there is activity, we can't see it. That the calorimter delivers signals is shown in the second plot. So this seems OK (except for the 3TeV tower itself).
<P>
2) Although the COT is switched off, there are COT hits - always in the same region. This can also be seen in the other consumers.
A typical occupancy plot for COT layer 1 is shown in the last picture. The plots for layers all 1, 4, 5, 8 look similar (with hits in the right-most quarter of the plot), while the plots for layers 2, 3, 6, 7 show no activity (as one would expect given that the COT is switched off).
<P>


<!-- Date: Wed Nov 28 05:08:28 2001 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Wed_Nov_28_2001_050816.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-144  align='top'></td><td>The plot to the 3TeV tower.</a></td></tr></table>
<!-- Author: Jonas -->


<!-- Date: Wed Nov 28 07:26:19 2001 -->
Having problems with XMon again - seem similar to the one <a href= 
'/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=4&frame=2&anchor=01:38:25'> 
seen earlier </a>. I'll leave it off for the time being.
<!-- Author: Jonas -->


<!-- Date: Wed Nov 28 17:03:07 2001 -->
<p>At the beginning of shift TrigMon consumer display was not running. I started it with the command "./startDiplay.sh TrigMon", but got the following message:<p> 
 
cdfdaq/cdf/b0dap51.fnal.gov@FN@b0dap52's password:<p> 
 
I typed in cdfdaq password, but it did not work, script asked for password again and then terminated with "Permission denied" message.<p> 
 
I have found, that the only difference between the current version of the script, startDisplay.sh, and the emacs backup copy, startDisplay.sh~, dated by November 8th, is the following string:<p> 
 
/usr/krb5/bin/kinit -k -t /var/adm/krb5/cdfdaq.keytab cdfdaq/cdf/`hostname`<p> 
 
So, I have copied startDisplay.sh to startDisplay_test.sh and then commented the above string in startDisplay.sh<p> 
 
After this modification startDisplay.sh runs as usual.<p>  
<!-- Author: Sasha -->


<!-- Date: Wed Nov 28 17:13:25 2001 -->
<p>The same problem, which I have mentioned in the previous entry, is about "startEvd.sh" script. I have copied<p> 
startEvd.sh to startEvd_test.sh<p> 
and then commented string with "kinit" command in "startEvd.sh" script.<p>
<!-- Author: Sasha -->


<!-- Date: Thu Nov 29 00:07:27 2001 -->
<p>Since all consumers crashed with new L3 trigger set PHYSICS_0_01[4,216], we have add the following string:<p>  
module disable PuffModule<p> 
at the very beginning of each consumer *.tcl file, except SiliMon, SVXMon and SVTSpyMon<p>
<!-- Author: Matt/Sasha -->


<!-- Date: Thu Nov 29 01:06:59 2001 -->
<pre>SiliMon keeps crashing with the following output: 
 
============================================= 
SiPatternRec: Creating 3 nominate, pursue and accept ti 
mers. 
SysError in &lt;TUnixSystem::UnixTcpService&gt;: bind (Addres 
s already in use) 
SysError in &lt;TUnixSystem::UnixTcpService&gt;: bind (Addres 
s already in use) 
DisplayServer: Connected successfully to CONSUMER socke 
t at port 9051 
                              on host b0dap67.fnal.gov. 
======================================================= 
==================== 
 
DisplayServer::openClientServerSocket(): Server socket  
for HISTO DISPLAY 
               clients at PORT 9092 available. 
 
               Use   b0dap67.fnal.gov:9092 
               as input stream to connect from the Hist 
oDisplay 
               to the Display Server. 
 
======================================================= 
==================== 
 
APPConsumerInputModule::initIOSystem ===&gt; CS_ConsSend p 
ort = 4031 
APPConsumerInputModule::initIOSystem ===&gt; CS_Receiver p 
ort = 4030 
%ERLOG-w DBNOTIFY: CotBeamPosition : cdf_reader/reader@ 
cdfofprd  
         asked to retrieve a calibration from the curre 
nt UsedSet, 
          
         no calibration for this table in the current U 
sedSet  CotBeamPosition  
         29-Nov-2001 01:01:14 run = 133667 event = 0 
Socket not valid. 
SiAlignmentManager: turning off alignments ... source = 
 none 
SiAlignmentManager at beginRun: no DB requested 
MostRecentSiCalibration:: DCMS mode is ON 
MostRecentSiCalibration:: DCMS offset set to 0 
MostRecentSiCalibration:: ChipStatus DB set to onotl_pr 
d_read 
doing unkeyed get 
 ave calib time = 305.742 
exiting CT_Cal class 
%ERLOG-w DBNOTIFY: COTCti : cdf_reader/reader@cdfonprd  
         asked to retrieve a calibration from the curre 
nt UsedSet, 
          
         no calibration for this table in the current U 
sedSet  COTCti  
         29-Nov-2001 01:01:28 run = 133667 event = 0 
exiting CT_Cal class 
CT_Calibration::_setDriftModelConstants: Using values i 
n database 
CT_Calibration::_setDriftModelConstants: Doing unkeyed  
get from DB  
CT_Calibration::_setDriftModelConstants: Found Run 1336 
65      Version 1       Status COMPLETE 
KalDetectorNode::set(): Unhandled Material "ALUMINUM in 
 physical volume CentralFlangePhys0"  (defaults to sili 
con.) 
%ERLOG-e makeJetAlgParams: :  
         /home/cdfsoft/dist/packages/JetObjects/V00-00- 
37/src/makeJetAlgParams.cc  
        : 275  could not match rcpid  29 2  Returning n 
ull pointer    
         GlobalLibraryLogger 29-Nov-2001 01:01:31 run = 
 133667 event = 7043 
%ERLOG-s Abnormal termination: caused by signal:  11 Fr 
amework abortJob()  
         29-Nov-2001 01:01:31 run = 133667 event = 7043 
Calling abortJob for DummyInput... 
Calling abortJob for FileInput... 
Calling abortJob for DHInput... 
</pre>
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 01:10:18 2001 -->
<pre>Trying to start L3RegionalMon doesn't do anything (no window comes up, no error message). 
 
This is all that happens: 
 
b0dap51:62:~/consumer_scripts&gt; ./startMon.sh L3RegionalMon 0 
  
This script should be executed by a Consumer Operator 
  
  
to view the results of consumer monitors, 
type: 
  
./startDisplay.sh All 
</pre>
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 01:18:20 2001 -->
<pre>Event display behaves badly. Sometimes it won't start (the text window just disappears), once we ended up with an AC++ prompt (there were some messages about a broken pipe in this case). 
Restarting it a few times usually does the trick for a while. When the run stopped, it crashed (it used to stay alive in that case). </pre>
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 01:28:25 2001 -->
Event display crashes on start-up (window disappears). I tried 5 times, and gave up.
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 01:55:21 2001 -->
<pre>&lt;h4&gt; startEvd error-log&lt;/h4&gt; 
I copied startEvd to jonas_startEvd and modified that script such that the output is saved in a file: evd_output (you can find it in the consumer_scripts area). The crucial lines appear to be: 
&lt;P&gt; 
Level-1 trigger bits set in TFRD after prescaling = 5 7 19  
Number of Level-2 triggers = 38 
Level-2 trigger bits set in TL2D after prescaling = 13  
Exit OK from APPConsumerInputModule::nextEvent 
%ERLOG-e makeJetAlgParams: :  
         /home/cdfsoft/dist/packages/JetObjects/V00-00-37/src/makeJetAlgParams.c 
c  
        : 275  could not match rcpid  29 2  Returning null pointer    
         GlobalLibraryLogger 29-Nov-2001 01:47:50 run = 133668 event = 129922 
%ERLOG-s Abnormal termination: caused by signal:  11 Framework abortJob()  
         29-Nov-2001 01:47:50 run = 133668 event = 129922 
Calling abortJob for DummyInput... 
Calling abortJob for FileInput... 
</pre>
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 02:55:48 2001 -->
<h4>ObjectMon</h4> 
 
ObjectMon produces hundreds and hundreds of lines of output that looks like this: 
 
<pre> 
In CdfJet: particle with p = 0 - could not return theta 
In CdfJet: particle with p = pz  - could not return eta 
In CdfJet: particle with p = pz  - could not return eta 
In CdfJet: particle with p = 0 - could not return theta 
In CdfJet: particle with p = 0 - could not return theta 
In CdfJet: particle with p = pz  - could not return eta 
In CdfJet: particle with p = pz  - could not return eta 
In CdfJet: particle with p = 0 - could not return theta 
In CdfJet: particle with p = 0 - could not return theta 
In CdfJet: particle with p = pz  - could not return eta 
In CdfJet: particle with p = pz  - could  
</pre>
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 03:51:43 2001 -->
 
I made a few copies of the event-display scripts and changed them such that they used a local copy of mon. This mon script I changed such that it had the PuffModule disabled. Then I ran the event display. I got further than before - the event display window came up briefly, but then stopped. The entire log is saved in the file evdOutput_without_puffmodule.log.  
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 04:01:16 2001 -->
<H4>Consumer-People: Scripts in consumer_scripts</H4> 
I played around in the consumer_scripts area, but don't worry, I did not modify any existing scripts. All modified copies of them have names starting with the five letter word jonas. Feel free to remove them all. However, maybe you find jonas_startEvd.sh useful. It is exactly like the original, except that the output gets saved and the output window doesn't disappear when there is a crash.<P>
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 06:51:44 2001 -->
<P>A further attempt at the event display problem:<P> 
I changed the TCL_TAG in start_local_evd.sh to 412 and got the same 
result as with the disabled puffMudule, i.e. there is progress, but shortly after the event display window appears the thing shuts down again. Have a look at at  
evdOutput_with_TCL_TAG=412.log. After this try, I changed the TCL_TAG back to what it was (362). 
  
<!-- Author: Jonas -->


<!-- Date: Thu Nov 29 18:05:06 2001 -->
<pre>Dear CO people, 
 
For the new L3 EXE, physics_0_02[4.216] table, 
please try startup scripts: 
 
./startMon_new.sh All(or consumer name)   &lt;partitionID&gt; 
 
and to startup event display, 
 
./startEvd_new.sh &lt;partitionID&gt; 
 
(histodisplay GUI does not matter.  you can start them 
with the usual, ./startDisplay.sh All 
===== 
 
I disabled SiliconClusteringModule which was crashing 
event display last night, so hopefully eventdisplay works 
tonight. 
 
</pre>
<!-- Author: kaori -->


<!-- Date: Thu Nov 29 20:59:24 2001 -->
Ming has told us that LumMon does not send luminosity inofmation to data catalog because it cannot connect to offline computers. The following message in LumMon window indicates this problem:<p> 
sh: dbPing: command not found<p> 
A modification should be done in the scripts which starts LumMon. Basically, one needs issue "setup cdfdb" command.<p>
<!-- Author: Sasha -->
<!-- Comment: Thu Nov 29 21;40;31 comment by...Sasha -->
<p>Kaori just called and told that she have fixed this problem. She add "setup cdfdb" into runLumMon script.<p>


<!-- Date: Fri Nov 30 02:11:54 2001 -->
<pre>We aren't taking any data at the moment and therefore the Consumers crashed. All the other consumers said good-bye in a decent way, while SVXMon got could in an infinit loop (Ctr-c won't stop it). Not a big deal, killing the window stops it, but maybe the Consumer-people find this interesting. This is the output: 
 
&lt;pre&gt; 
         datetime: Fri Nov 30 02:09:20 2001 
         src file: /cdf/people1/csl/temp/cs-logger/src/library/l3_csl_socket.c 
         src line: 880 
         src info: send() -&gt; broken socket connection 
         sys perr: Broken pipe 
 
-------------------------------------------------------------------------------- 
ERROR--&gt; hostname:  
         prg name: anonymous 
         function: l3_csl_socket_send() 
         datetime: Fri Nov 30 02:09:21 2001 
         src file: /cdf/people1/csl/temp/cs-logger/src/library/l3_csl_socket.c 
         src line: 880 
         src info: send() -&gt; broken socket connection 
         sys perr: Broken pipe 
 
-------------------------------------------------------------------------------- 
ERROR--&gt; hostname:  
         prg name: anonymous 
&lt;/pre&gt;</pre>
<!-- Author: Jonas -->


<!-- Date: Fri Nov 30 05:17:31 2001 -->
<h4>CEM</h4> 
o Eta-tower -8, phi-wedge 16 shows very high reading for the average raw ADC count per hit (~500, while typical values are around 100). 
<BR>o Eta-tower 7, phi-wedge 5 constantly gives the highest occupance-per-event readings (less dramatic: 0.06 while typical values are 0.04)
<!-- Author: Jonas -->


<!-- Date: Fri Nov 30 16:01:44 2001 -->
<pre>Hi Kaori, 
the shot setup is in progress and I tried to restart 
all consumers to be ready for shot. 
I asked to the Ace and L3 experts and they told me 
that we are using L3 linked with Offline version 4.1 
As you told me this morning I started the monitors 
with  
./startMon_new.sh All 0 
 
I also reset all Display. 
While Ymon/Objectmon and others are running , there are 
XMON,TRIGMON,BEAMMON which keep crashing. 
I also tried to reach you by paging you. Please call 2080. 
		Ciao 
			Stefano</pre>
<!-- Author: Miscetti -->


<!-- Date: Fri Nov 30 17:34:30 2001 -->
<pre>From YMon and DAQ mon. 
run 133896  
high occupancy in Crate COT-15 (86%) cf DAqmon and Ymon pic included</pre>
<!-- Author: Antonio -->


<!-- Date: Fri Nov 30 22:47:43 2001 -->
<pre>Instructions to run EventDisplay 
from an e-mail from Anrei to Kaori 
 
I've made a tcl file - monitor.tcl, it is in the 
 
/cdf/people1/cdfdaq/consumer_scripts. 
 
To run EventDisplay, do the following: 
 
mon.exe /cdf/people1/cdfdaq/consumer_scripts/monitor.tcl 
 
One more problem is if one tries to save the picture from the display, 
he/she should use "Save As canvas.gif" (or canvas.eps, canvas.ps, canvas.root, canvas.C) in the save section. 
There will be an output in the xterm window what file was created.</pre>
<!-- Author: Antonio -->
<!-- Comment: Sat Dec  1 02;20;57 comment by...Igor, Jonas -->
<pre>Before doing that, you need to

source ~cdfsoft/cdf2.cshrc
setup cdfsoftb0

</pre>
<!-- Date: Sat Dec 1 02:19:42 2001 -->
<pre>Could not restart BeamMon: got the following error messages 
 
1 
Dec  1 02:08:48 2001 Opening connection to database &lt;cd 
fonprd.fnal.gov&gt; 
Dec  1 02:08:48 2001Accessing data base string: 
INSERT INTO CONSUMER_SUMMARY (RC_RUNNUMBER,NAME,CID,PRO 
CEVT,EVALUATION,COMMENTS) VALUES (133908,'BeamMon',CONS 
UMER_SEQUENCE.NEXTVAL,636,'BAD','d-phi:db_not_filled..v 
x-cot:db_not_filled;') 
Error in &lt;TFile::TFile&gt;: file /data1/consumer/results/B 
eamMon/BeamMon133908aa.root already exists 
*** Error OCI_ERROR  [-1] [1] 
*** Error: ORA-00001: unique constraint (RUNDB_PRD.CNS_ 
UNIQUE) violated 
BeamMon: Root file /data1/consumer/results/BeamMon/Beam 
Mon133908ab.root saved. 
BeamMon: summary file /data1/consumer/results/BeamMon/B 
eamMon133908.sum written. 
DisplayServer::pollConsumer(): INFO: BeamMon sent messa 
ge: END CONNECTION 
 
 
 
Copied the BeamMon log file to 
/data1/consumer/runarea/BeamMon/BeamMon.cdfdaq_133908.log 
</pre>
<!-- Author: Igor, Jonas -->



<!-- Date: Sat Dec 1 02:33:41 2001 -->
<table><tr><td><a href=/elog/notebooks/consumer-ops/Sat_Dec__1_2001_023323.ps align=top><img src=/cgi/elog/det-system-elog.pl?nb=consumer-ops&action=view&page=-146  align='top'></td><td>Hot CEM Tower (East 3, phi wedge 15). Seems to persist for at least few runs (looked at 133904, 133908) </a></td></tr></table>
<!-- Author: Igor, Jonas -->
<!-- Comment: Sat Dec  1 02;43;01 comment by...Jonas -->
<pre>Also:

CEM PMT Occupancy is high in tower 3 Phi-wedge 15R   
CHA PMT Occupancy is high in tower 2 phi-wdge 10L
WHA PMT Occupancy is high in tower 9 (east), Phi wedge 8R,
</pre>


<!-- Date: Sat Dec 1 03:16:34 2001 -->
ObjectMon keeps crashing. 
Copied log file to:<BR> /data1/consumer/runarea/ObjectMon/ObjectMon.cdfdaq_crash_at_run_133908.log  
<!-- Author: Igor, Jonas -->