<!-- Title: 08:14:52  Tue Dec 10 2002 -->
<!-- Crewchief: new day -->
<!-- Op1: &nbsp; -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>database activity</pre>


<!-- Date: Tue Dec 10 08:15:13 2002 -->
<pre>refresh of int from prd --&gt;  app's unavailable all day for INT </pre>
<!-- Author: nstanfield -->
<!-- Comment: Tue Dec 10 10;46;49 comment by...nstanfield  -->
<pre>STATUS:

Refresh on INT with PRD Complete :  cdftrig & hdwdb 

Refresh on INT with PRD IN PROCESS:   run & mcs(slow controls)
Expect it to complete at NOON or so.  

Yet to do..  calib.  Will start this afternoon.



</pre>
<!-- Comment: Tue Dec 10 16;33;03 comment by...ns -->
<pre>UPDATE

MCS completed but the sensor histories tables failed
because it needed to grab 2.1 G.. the datafile was only 2G... it is now larger and JUST this table is being imported.


RUN Still running still 10:30..  tables look ok but 1
of the indexes failed and it's still going...

have not started calib...

alert from patrol

b0dau36.fnal.gov - cpu

--------------------------------------------------------------------------------
 
red Tue Dec 10 16:24:23 CST 2002 up: 21 day(s), 6 users, 133 procs, load=795 


LOAD AVG on b0dau36,fnal,gov is 795

Status unchanged in 7.68 hours
Status message received from 131.225.236.186 
</pre>
<!-- Comment: Wed Dec 11 10;16;48 comment by...nstanfield  -->
<pre>toolman back on with resource limit to 2 on cdfonint.
imports still continue.. had a few tables fail!
sensor_histories for slow controls and messages for run..
they are running now... DB NOW in NO archive too..

have not even started CALIB!</pre>
<!-- Date: Tue Dec 10 09:25:39 2002 -->
<pre>
Target Name : b0dau35.fnal.gov 
Target Type : Node 
Node Name : b0dau35.fnal.gov 

CPU Utilization 
The CPU utilization is high.  Current utilization: 92. 
CRITICAL 
Dec 10 2002 09:21:28 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Tue Dec 10 10:10:31 2002 -->
<pre>
Target Name : b0dau35.fnal.gov 
Target Type : Node 
Node Name : b0dau35.fnal.gov 

CPU Utilization 
Event test "CPU Utilization" is cleared 
CLEARED 
Dec 10 2002 10:06:31 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Tue Dec 10 10:34:14 2002 -->
<pre>
</pre>
<!-- Author: oem-admin (email) -->





<!-- Date: Tue Dec 10 17:09:04 2002 -->
<pre>&gt;  The following Oracle Session has been Terminated 
&gt; 
&gt;  Data  Base  : cdfonprd 
&gt;  OS PID : 15467 
&gt;  DB Account : CDF_VIEWS_PRD 
&gt;  OS Account : root 
&gt;  Machine  : b0dau30 
&gt;  Program  : sqlplus@b0dau30 
&gt;  Logon Time  : 12/10/200215:05 
&gt; 
&gt; Tue Dec 10 16:05:04 CST 2002 
&gt; 
 
 
Hi Rick, 
 
This problem with the DBbroker has brought back to 
the surface the fact that you're using CDF_VIEWS_PRD 
account in a frequently accessed, read-only application. 
 
CDF_VIEWS_PRD is something like an "application owner" in that 
it has lots of special privileges so that it can create views, etc. 
Therefore it should not be used for read-only applications with 
the password embedded, but only in controlled interactive sessions. 
 
Can you please change DBbroker to use CDF_READER account 
instead?   Should be a trivial and harmless change. 
 
Thanks, 
Bill 
</pre>
<!-- Author: nstanfield  -->

<!-- Date: Tue Dec 10 17:30:46 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 

Tablespace Full 
Tablespace RUNDB_DATA is 83.86% full.  1.33894e+06 KBytes of free space remain. Tablespace MCS_INDEX is 80.13% full.  2.7056e+06 KBytes of free space remain. 
WARNING 
Dec 10 2002 05:26:19 PM 


</pre>
<!-- Author: oem-admin (email) -->
<!-- Comment: Wed Dec 11 09;33;58 comment by...akumar -->
<pre>There is not  much available to be allocated to RUNDB_DATA and MCS_INDEX  on cdfonprd.
Mount points are already at 98-99% full.

/dev/vx/dsk/oradg/mcsindvol 17976320 17512844  456240    98%          /dbs15
/dev/vx/dsk/oradg/rundbvol 8388608 8297645   85283           99%           /dbs16

However, we have 2.7GB available for MCS_INDEX and 1.3GB for RUNDB_DATA

b0dau35&gt; runscript cdfonprd show_tablespace_usage | grep MCS_INDEX
MCS_INDEX              13,946,060,800    11,175,534,592    2,770,526,208        81
b0dau35&gt; runscript cdfonprd show_tablespace_usage | grep RUNDB_DATA
RUNDB_DATA              8,493,465,600     7,122,386,944    1,371,078,656        84


If these tablespaces will hit 90% before Jan shutdown
then we have to make  contingency allocation, otherwise during Jan shutdown more disk space will be
available for allocation.


Thanks,

Anil.


oem-admin@fnal.gov wrote:

&gt; Name : cdfonprd_tablespaces
&gt; Target Name : cdfonprd.fnal.gov
&gt; Target Type : Database
&gt; Node Name : b0dau35.fnal.gov
&gt;
&gt; Tablespace Full
&gt; Tablespace RUNDB_DATA is 83.86% full.  1.33894e+06 KBytes of free space remain. Tablespace MCS_INDEX is 80.13% full.  2.7056e+06 KBytes of free space remain.
&gt; WARNING
&gt; Dec 10 2002 05:26:19 PM
</pre>


<!-- Date: Wed Dec 11 09:46:18 2002 -->
<pre>Alert on CPU utilization on fcdflnx1.    User using most 
cpu is 'munar'.  
 
 
Name : fcdflnx1_cpu 
Target Name : fcdflnx1.fnal.gov 
Target Type : Node 
Node Name : fcdflnx1.fnal.gov 
Status : WARNING 
 
CPU Utilization 
The CPU utilization is high.  Current utilization: 79. 
WARNING 
Dec 11 2002 09:40:02 AM 
 
 
 
  TIME %CPU   PID USER     PRI  NI  SIZE  RSS SHARE STAT %MEM COMMAND 
 15:16 52.7 12063 oracle    17   0 18308  17M  8792 R     0.4 oracle 
 14:57 46.0 12797 oracle    16   0 18312  17M  8796 R     0.4 oracle 
 15:56 44.1 11971 oracle    15   0 21284  20M 11744 R     0.5 oracle 
 14:16 36.4 13397 oracle    16   0 18320  17M  8804 S     0.4 oracle 
  0:01 11.5 19841 oracle    17   0 22800  22M 11680 S     0.5 oracle 
  4:13  2.8 28591 oracle    15   0 53216  50M 10496 R     1.2 oracle 
 
[oracle@fcdflnx1 ~]$ runscript cdfrep01 show_users | grep 12063 
12063           85   44352 CDF_READER   munar        fcdflnx2.fnal.gov    SecVtxTest@fcdflnx2.fnal.gov (TNS V1-V3) 
[oracle@fcdflnx1 ~]$ runscript cdfrep01 show_users | grep 12797 
12797           10   35509 CDF_READER   munar        fcdflnx2.fnal.gov    SecVtxTest@fcdflnx2.fnal.gov (TNS V1-V3) 
[oracle@fcdflnx1 ~]$ runscript cdfrep01 show_users | grep 11971 
11971           77   53723 CDF_READER   munar        fcdflnx2.fnal.gov    SecVtxTest@fcdflnx2.fnal.gov (TNS V1-V3) 
[oracle@fcdflnx1 ~]$ runscript cdfrep01 show_users | grep 13397 
13397           87    6013 CDF_READER   munar        fcdflnx2.fnal.gov    SecVtxTest@fcdflnx2.fnal.gov (TNS V1-V3</pre>
<!-- Author: nstanfield -->
<!-- Comment: Wed Dec 11 10;16;07 comment by...nstanfield  -->
<pre>Name : fcdflnx1_cpu
Target Name : fcdflnx1.fnal.gov
Target Type : Node
Node Name : fcdflnx1.fnal.gov
Status : CLEARED

CPU Utilization
Event test "CPU Utilization" is cleared
CLEARED
Dec 11 2002 10:10:01 AM
</pre>


<!-- Date: Wed Dec 11 10:33:33 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Wed Dec 11 10:33:33 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Wed Dec 11 10:33:33 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Wed Dec 11 10:34:30 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->



<!-- Date: Wed Dec 11 11:57:59 2002 -->
<table border=0><td><tr><td bgcolor=#e0e030><pre>For Rick Vidal, ran commands: 
 
CREATE OR REPLACE PUBLIC  SYNONYM SIFIBPED FOR SIFIBPED 
 
CREATE OR REPLACE PUBLIC  SYNONYM SITEST FOR SITEST 
 
on cdfondev, cdfonint, cdfonprd 
 
I don't think they're needed offline, but am not sure. 
 
</pre></td></tr></td></table>
<!-- Author: W.Badgett -->


<!-- Date: Wed Dec 11 12:34:29 2002 -->
<pre>Turned on the logic for CPU_usage for each session on cdfofprd.  
This version of trigger is running on cdfrep01 . And query is 
not taking temp tablespace. It is tuned version of trigger.</pre>
<!-- Author: akumar -->


<!-- Date: Wed Dec 11 12:51:51 2002 -->
<pre>I have completed refreshing the cdfofint SAM schema with production data from 11-Dec-2002.  </pre>
<!-- Author: diana -->


<!-- Date: Wed Dec 11 14:45:07 2002 -->
<pre>We were getting ORA-00942 errors, with "table or view v$session does not exist", from the development version of the CDF Software today (but from neither the 4.9.1 nor the 4.9.0int2 versions).  This error is from the "Database Monitoring Patch" software.  Anil has granted SELECT_CATALOG_ROLE to Oracle account FILECATALOG_READUSER on database instances cdfrep01 and cdfofprd.  This solved the problem. </pre>
<!-- Author: Eric Wicklund -->


<!-- Date: Wed Dec 11 16:38:09 2002 -->
<pre>Started the refresh of CALIB schema at 3:25PM 12/11/02  
on cdfonint with production data </pre>
<!-- Author: akumar -->

<!-- Date: Thu Dec 12 01:34:03 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1652: unable to extend temp segment by 321 in tablespace 		USERTEMP_01. 
WARNING 
Dec 12 2002 01:28:33 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Thu Dec 12 01:42:21 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1652: unable to extend temp segment by 321 in tablespace 		USERTEMP_01. 
WARNING 
Dec 12 2002 01:38:35 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Thu Dec 12 02:03:06 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1652: unable to extend temp segment by 321 in tablespace 		USERTEMP_01. 
WARNING 
Dec 12 2002 01:58:39 AM 


</pre>
<!-- Author: oem-admin (email) -->



<!-- Date: Thu Dec 12 10:09:42 2002 -->
<pre>The following query was heavy temp_space usage query.  
Temp area ran out of space due to this query.   
 
 
CDF_READER   snihur          fcdflnx3.fnal.gov      66    3207 USERTEMP_01        SORT        SELECT     'OFFLINE_STATUS = 
                                                                                             1                 ' "REQUIREM 
                                                                                            ENT",     SUM(rc.B0INTEGRATEDL 
                                                                                            UMI) IntLumi,     SUM(rc.B0INT 
                                                                                            EGRATEDLIVELUMI) IntLiveLumi, 
                                                                                                SUM(rc.CSL_NEVENTSOUTPUT) 
                                                                                            CSLOUT,     SUM(rc.CSL_EVENTSU 
                                                                                            NIQUE) CslOutUnique,     MIN(r 
                                                                                            c.RUNNUMBER) MINRUN,     MAX(r 
                                                                                            c.RUNNUMBER) MAXRUN,     COUNT 
                                                                                            (rc.RUNNUMBER) RUNSCNT    FROM 
                                                                                             RUNCONFIGURATIONS rc, RUNSETS 
                                                                                            , DATATYPES, PHYSICSTABLES 
                                                                                            WHERE rc.RS_NAME = RUNSETS.NAM 
                                                                                            E      AND RUNSETS.DATATYPE = 
                                                                                            DATATYPES.DATATYPE      AND rc 
                                                                                            .OFFLINE_STATUS = 1 
 
 
 
This query need to be tuned. </pre>
<!-- Author: akumar -->

<!-- Date: Thu Dec 12 10:34:28 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Thu Dec 12 10:34:28 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Thu Dec 12 10:34:28 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Thu Dec 12 10:34:28 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->



<!-- Date: Thu Dec 12 10:45:01 2002 -->
<pre>Restarted the CALIB refresh on cdfonint from PRD at 9:15AM 
Yesterday refresh job was killed because connection got  
terminated. Now job is running with nohup command. Connection  
termination will have no impact.  
</pre>
<!-- Author: akumar -->


<!-- Date: Thu Dec 12 16:54:24 2002 -->
<pre>Refresh of CALIB on cdfonint aborted because export dump got  
deleted by some cron job between 12:00PM - 1:00PM  
Made sure it was not a cron job by oracle user.  
Send e-mail to System Admins to find out if they are aware of  
any cron job which deletes the export dump from /backup/export area.  
 
Started the full export agains. Then we will start all over again 
for CALIB refresh.  
 
Index for Slow control still going. </pre>
<!-- Author: akumar -->