<!-- Title: Eve shift. Mon Jan 22, 2001 -->
<!-- Leader: Frank Chlebana -->
<!-- Ace: Kirby & Tracey Pratt -->
<!-- Co1: &nbsp; -->
<!-- Co2: &nbsp; -->
<!-- Co3: &nbsp; -->
<!-- Co4: &nbsp; -->
<!-- Co5: &nbsp; -->
<!-- Notes: -->
<pre>Starting Shift and looking on COT crates and rate tests through L3, and general large scale integration.</pre>


<!-- Date: Mon Jan 22 16:14:34 2001 -->
<pre>Found the problem we had with loading the Et weighting  
algorithm via run control.  
 
The algorithm is first loaded to FRAM then one has to  
toggle a program bit. The FPGA can also be loaded if the crate 
is reset. 
 
Run control will now toggle the load bit so the new version 
should be loaded. 
 
After this bit is toggled I can no longer access some registers 
on the ADMEM and the cold/warm start will fail. You have to  
reboot the crates after configuring them with the new Et weighting algorithm. 
 
Someone should verify if we can access the registers after  
we configure the Et algorithm via qietest.... 
 
Run 106122 25K L1 rate 24Hz taken with CCAL00,01,02  
Version 7DEC00 of the Et weighting algorith, 
 
Run 106131 19K L1 rate 68Hz same configuration. 
Version v3 of the Et weighting algorithm. 
 
Both runs were taken with the software event builder. 
 
These runs should be used to validate the Et algorithm  
versions... 
 
At least the rates are now different... </pre>
<!-- Author: Frank Chlebana -->
<!-- Comment: Mon Jan 22 21;28;20 comment by...carla -->


Check of the ET summing algorithm:

 	run 106122 : sum of average of two pmts

	run 106131 : sum of four pmts

The tower granularity in Dirac is still not the correct one.

has the value been changed in the proper 
database?

<!-- Comment: Tue Jan 23 08;40;26 comment by...Simona -->
I've run TRGSim on the 2 runs: 
106122 - The factor of 2 between DCAS real and simulated disappeared.
It looks like the ET summing is done properly.
There is still disagreement in Cratesum, but it's due to the wrong setting of  TOW_GRAN

106131 - The factor of 2 between real and simulated DCAS is there, as previously. with the simulation smaller.
Example:
 -- event number =7 eta = 8, phi = 0
had_energy_sim[word=0][card=1][block=0] = 3 --   had_energy_Real[word=0][card=1][block=0] = 7
 -- event number =7 eta = 9, phi = 1
had_energy_sim[word=1][card=4][block=0] = 2 --   had_energy_Real[word=1][card=4][block=0] = 5
 -- event number =7 eta = 8, phi = 2
had_energy_sim[word=0][card=7][block=0] = 18 --   had_energy_Real[word=0][card=7][block=0] = 31

 -- event number =8 eta = 9, phi = 4
had_energy_sim[word=1][card=1][block=1] = 2 --   had_energy_Real[word=1][card=1][block=1] = 4
 -- event number =8 eta = 10, phi = 4
had_energy_sim[word=2][card=1][block=1] = 2 --   had_energy_Real[word=2][card=1][block=1] = 3
 -- event number =8 eta = 8, phi = 1
had_energy_sim[word=0][card=4][block=0] = 0 --   had_energy_Real[word=0][card=4][block=0] = 1
 -- event number =8 eta = 9, phi = 1
em_energy_sim[word=1][card=4][block=0] = 0 --   em_energy_Real[word=1][card=4][block=0] = 1
 -- event number =8 eta = 9, phi = 1
had_energy_sim[word=1][card=4][block=0] = 5 --   had_energy_Real[word=1][card=4][block=0] = a
 -- event number =8 eta = 9, phi = 2
had_energy_sim[word=1][card=7][block=0] = e --   had_energy_Real[word=1][card=7][block=0] = 1b
 -- event number =8 eta = 9, phi = 3
had_energy_sim[word=1][card=10][block=0] = 8 --   had_energy_Real[word=1][card=10][block=0] = f

I infere that the ET summing is not working properly for this run.

<!-- Date: Mon Jan 22 16:18:27 2001 -->
QIE calibration for the entire Central Cal.
<!-- Author: Frank Chlebana :: (run 106136) -->


<!-- Date: Mon Jan 22 16:20:06 2001 -->
<pre>Run taken after calibration. 
Used all the central crates and calorimeter trigger. 
 
About 5000 events. 
 
Use this run to check if teh FRAM is being loaded properly... 
Look at a plot of the pedestals... 
</pre>
<!-- Author: Frank Chlebana :: (run 106138) -->


<!-- Date: Mon Jan 22 17:24:11 2001 -->
<pre>Tried to start a run using the Endwall crates, muon trigger, and l1 and l2 trigger crates and scalers.  The new thing was including TOFD banks in the endwall readout.  The one wcal crate not included was b0wcal02.  This crate has been powered off for a reason unknown to us.  Had trouble once the run started, it halted after ~41 event giving warnings in the green screen of death.  Saying that the number of data words that the VRB saw was not consistent with the number of data words the front end code, rather violently.  So this caused the system to crashed.  Removing the muon trigger crates, caused this problem to go away, and the run went well for about 1000 events.</pre>
<!-- Author: Kirby -->


<!-- Date: Mon Jan 22 17:27:59 2001 -->
<pre>Looking at the event display, the events had abnormally high admem values in the wcal.  So we decided to take a calibration run.  During the calibration run, we were logged into wcal00 and noticed that there was a large number of print out statements.  Not sure where these are coming from, so all of the wcal crates (except for wcal02) were rebooted and the calibration was done again.  This new calibration was run 106160.  Note that after the reboot, wcal00 didn't seem to show any problems with large print out statements.</pre>
<!-- Author: Kirby :: (run 106160) -->
<!-- Comment: Mon Jan 22 17;37;39 comment by...Kirby -->
After the second calibration the pedestals appeared reasonable and the data looked much better.  ran for ~19000 events without any problems, and the data rate determined by deadtime from L3 since we were only using 1 subfarm.  I like traffic lights.


<!-- Date: Mon Jan 22 17:46:01 2001 -->
<pre>Tried to use the same configuration, but included the cmu00 and cmu01 and got the same problem as we had orignially, with the run stopping very shortly after starting (~18).  The TRACER count of data words was not the same as what the VRB counted.  This was run (106164)  so Frank changed the configuration from SPY mode to non-SPY mode and the problems went away. also, we added in 3 more L3 subfarms for a total of 4 (0,4,8,12).</pre>
<!-- Author: Kirby :: (run 106164) -->


<!-- Date: Mon Jan 22 18:26:36 2001 -->
<pre>Did a run with all of the Cal except pcal03 and wcal02 (accidentally left out) using the sumEt trigger and 4 subfarms.  This ran for 3000 events without problems.  Stopped the run in order to include several cot crates and to put in wcal02.  </pre>
<!-- Author: Kirby :: (run 106168) -->


<!-- Date: Mon Jan 22 18:37:47 2001 -->
<pre>Took a run including all of the cal except for pcal03 using the SumEt trigger and L3 tag set 68 (no filtering)  The run was ~32000 events and run 106172.  We also used 4 subfarms at L3 and had a through put of 233Hz with very little dead time.  Tracey's first succesful run!</pre>
<!-- Author: Kirby :: (run 106172) -->


<!-- Date: Mon Jan 22 18:48:50 2001 -->
<pre>Trying to start a new run including cot18 and cot19, had trouble with b0cot18 slot 18 during the ColdStart transition.  The card was unavailable and didn't show up during a crate map.  Issued a crate bus reset and the card came back.  Restarted the run, and the card in slot 18 of cot18 didn't give any complaints.  Running with the cal_1 trigger tag set 73, and all of the cal crates plus 2 cot crates.  The readout rate is 233Hz, with all of the L3 farms busy.  The dead time was 100%, with 80% coming from from BUSY, and 20% from the readout.</pre>
<!-- Author: Kirby :: (run 106174) -->


<!-- Date: Mon Jan 22 19:00:05 2001 -->
<pre>this run has the same configuration as the previous run, but this time using all of the L3 subfarms except for 15 which has been reserved for use by L3 experts.   
 
A random aside:  During a cold start transition, the Warm Start button was not disable during the transition.  not sure why? 
 
So using all of the subfarms, the readout from L3 is not being saturated by a BUSY status on the farms.  the readout rate is 370Hz, with the dead time at 100% and the break down being 60% BUSY and 40% readout.  The trigger is gggggrrrreat!  Using the sumEt trigger and L3 tagset 73.  With the thresholds set high on the COT, and the cables plugged in, there are very few hits in the COT or noise.</pre>
<!-- Author: Kirby :: (run 106178) -->


<!-- Date: Mon Jan 22 19:19:41 2001 -->
<pre>Run 106180 had 6 additional COT crates (6,7,16,17,18,19)  but on cold start 2 of the crates had trouble configuring the TDCs.  the cards that were unavailable during the cold start were: 
   cot16, slot 6 & 13 
   cot06, slot 20 
We tried sending a bus reset to these crates and the card in cot06 came back, while in cot16 the card in slot 6 responded, but the card in 13 did not.  So these crates were removed from the run and put into another run control in order to do tests on the TDC stability.</pre>
<!-- Author: Kirby :: (run 106180) -->
<!-- Comment: Mon Jan 22 19;26;50 comment by...Kirby -->
<pre>Had more problems with TDC hangs during the cold start.  are removing all of the COT crates from the readout and testing them in another partition.  The other hang was in b0cot19, slot 17.  this did recover after a crate reset.</pre>


<!-- Date: Mon Jan 22 19:33:07 2001 -->
<pre>During the process of trying to setup a run using all of the cal crates and the two cmu crates ( no COT)  there were problems in the process of releasing the COT crates.  during partitioning, the cot crates didn't appear in the acknowledgement window, but were not released from the resource manager. ??????</pre>
<!-- Author: Kirby :: (run 106183) -->
<!-- Comment: Mon Jan 22 19;37;08 comment by...Kirby -->
<pre>This run was using the Integration_Table_Sumet_1[1,68] physics trigger table.  All of the L3 subfarms (except 15) were used and a readout rate of 232 Hz out of the VRB.  While the L1 rate was ~500Hz, with about 50% deadtime with 80% due to BUSY and 20% due to readout.  The problem is that the L3 was BUSY on output.  But this is understandable since L3 was not rejecting anything and shows our limit going through the CSL and getting data to Feynman.</pre>


<!-- Date: Mon Jan 22 19:43:49 2001 -->
<pre>Same configuration, but added back in cot07,cot17,cot18,and cot19.  also, changed the L3 executable so that we're actually filtering some of the events and hopefully not completely saturating the output of L3.  </pre>
<!-- Author: Kirby :: (run 106184) -->
<!-- Comment: Mon Jan 22 19;47;57 comment by...Kirby -->
<pre>The readout rate was drastically lower than without the COT crates.  The rate was ~100Hz, with 100% of the dead time coming from readout.  All four of the crates added were readout in global mode, but we changed the readout to aggressive polling.</pre>

<!-- Comment: Mon Jan 22 19;49;53 comment by...Kirby -->
<pre>Noticed that in b0cot07, there were errors of data truncated coming from the frontend.  This may have been what was causing the slow readout.</pre>


<!-- Date: Mon Jan 22 19:51:34 2001 -->
<br><b>Masked Trigger Towers:</b> 
<br>Masked the following trigger towers: 
<pre> 
   eta  phi  cal 
  -------------- 
    6  7-10   EM 
</pre> 
These trigger towers triggered every event, but with no attending 
energy in the PEM.  Likely due to unplugged cables. 
<br>
<!-- Author: R.J. Tesarek -->


<!-- Date: Mon Jan 22 19:55:52 2001 -->
<pre>During the coldstart transition for the run with the new aggressive polling TDC readout (extra 4 = 999), but had problems with two cards.  b0cot17, slot 6 and b0cot19, slot 17 (had seen before) were unresponsive and so the coldstart failed.  Mapping the crate these cards had hung, but upon a crate reset they reappeared.  Hopefully, they had a good holiday and are now well rested.</pre>
<!-- Author: Kirby :: (run 106185) -->
<!-- Comment: Mon Jan 22 20;02;43 comment by...Kirby -->
<pre>Had problems with the TS and failed the activate transition due to "Ts failed when trying to Run (State 2)"  not good.</pre>


<!-- Date: Mon Jan 22 20:20:23 2001 -->
<pre>Again, more problems with the cot crates during a coldstart.  b0cot18, slot 18 and b0cot19, slot 17  both of which came back from a crate reset.   
There was also a problem with runControl where a java error was reported.  the error was: 
Exception occured during event dispatching: 
java.lang.OutofMemoryError 
 
runControl continued to work, but twice gave these errors.  not sure exactly which java process gave the error, since RC was in the background.</pre>
<!-- Author: Kirby :: (run 106190) -->
<!-- Comment: Mon Jan 22 20;24;20 comment by...Kirby -->
<pre>Still getting data truncation errors from b0cot07.  and is keeping readout down around 100Hz.</pre>


<!-- Date: Mon Jan 22 21:14:03 2001 -->
<font color=red>Shift Summary: </font><pre>After trying several times to start a run using the calorimeter and 3 cot crates (b0cot17,18,19) and failing all attempts, it was decided to return and fight another day.  There were problems with L3 during the cold start that Ilya was here to witness and was unable to diagnose completely.  they went away after an abort and reset, and a few magic incantations by Ilya.  the other problem stemmed mostly from b0cot19, and the card in slot 17.  during the shift this card hung 6 times.  4 times during a cold start and twice during the activate transition.  These problems kept us busy for about the last 30 minutes of the shift and so stopped us from getting any significant run at a high rate (>200Hz) for testing out the new firmware in the VRBs.  Thank you and goodnight, we'll be here all week.</pre> 
<br><b>End of Shift Numbers</b>
<table><tr><td width=60% align=left>
<b>CDF Run II <p></b>                       
<pre>
Runs                   106160 -> 106192
Delivered Luminosity   nada  pb-1
Acquired Luminosity    nichts  pb-1
Efficiency             100%
   
</td></tr></table></pre>
<!-- Author: Kirby & Tracey -->