<!-- Title: 08:26:01  Fri Nov 22 2002 -->
<!-- Crewchief: New Day  -->
<!-- Op1: &nbsp; -->
<!-- Op2: &nbsp; -->
<!-- Op3: &nbsp; -->
<!-- Op4: &nbsp; -->
<!-- Op5: &nbsp; -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<pre>database activity</pre>


<!-- Date: Fri Nov 22 08:47:22 2002 -->
<pre>Turned off toolman for NOW.   There were several there.  
 
 
Name : cdfofprd_processes 
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 
 
Process Limit 
The process usage is reaching its limit.  Current value: 165, limit: 200. 
CRITICAL 
Nov 22 2002 08:03:14 AM 
</pre>
<!-- Author: nstanfield  -->
<!-- Comment: Fri Nov 22 08;48;43 comment by...nstanfield  -->
<pre>investigating further.. art kreymer already investigating..


Your "cron" job on fndaut1
. /usr/local/etc/setups.sh && setup -qweb_prd sam_admin && samadmin sam at a glance --output=${SAM_WEB_LOCAL}/SamAtAGlance/index.html --server-list-file=/cdf/sam/private/fndaut1_server_list.txt --env-pattern="prd"

produced the following output:

OCI_ERROR at oci_8.c:325
ORA-00020: maximum number of processes (200) exceeded
Traceback (most recent call last):
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/bin/samadmin", line 19, in ?
    sys.exit( SamAdminCommandDefinitionObj.main(sys.argv) )
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/SamAdminCommandDefinitionObj.py", line 775, in main
    returnStatus = theCommand.dispatch()
  File "/cdf/ups/prd/sam_common/v4_2_1/NULL/src/python/CommandParser.py", line 504, in dispatch
    exec dispatchCode
  File "&lt;string&gt;", line 1, in ?
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/SamAtAGlance.py", line 769, in saagDispatch
    status = NameServiceObj.nameService_dispatchHandler(argDict, [])
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/NameServiceObj.py", line 897, in nameService_dispatchHandler
    ns.finish()
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/NameServiceObj.py", line 843, in finish
    self.action.finish(self.getProcessedServerCnt())
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/SamAtAGlance.py", line 503, in finish
    saagWriter.processSamStationInfo( self.getAllStationsSorted(), self.stationsSeenInNS )
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/SamAtAGlance.py", line 447, in getAllStationsSorted
    allStations = self.getAllStations()
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/SamAtAGlance.py", line 440, in getAllStations
    q = DbQueryObj.DbQueryObj()
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/DbQueryObj.py", line 87, in __init__
    self.connect()
  File "/cdf/ups/prd/sam_admin/v4_2_16/NULL/src/python/DbQueryObj.py", line 129, in connect
    raise DbQueryException("ERROR: cannot connect to database: %s" % exc.dbMessage)
DbQueryObj.DbQueryException: ERROR: cannot connect to database: ORA-00020: maximum number of processes (200) exceeded
</pre>
<!-- Date: Fri Nov 22 08:47:45 2002 -->
<pre>The cdfofprd database is being heavily loaded by the present farms processing. 
 
There are over 100 active connections, most from the farm, 
some of which have been connected and active for over an hour. 
Normally farm connections are active for about a minute, 
and may stay connected but idle for 5 minutes. 
But the farms seem to be running the standard 4.8.4a executeable. 
 
The root cause of this may involve a large number of active connections from 
user menzemer at nodes with names like c01-004-112,  at Karlsruhe. 
 
The cdfofprd database server has a load average of 40 right now, 
with all memory in use, and more than half the swap space in use. 
It is so badly loaded that even lightweight sqlplus queries 
are taking many minutes to perform. 
 
Please stop the memzemer jobs at Karlsruhe, while we investigate. 
</pre>
<!-- Author: akreymer  -->


<!-- Date: Fri Nov 22 09:04:54 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Process Limit 
Event test "Process Limit" is cleared 
CLEARED 
Nov 22 2002 09:03:16 AM 


</pre>
<!-- Author: oem-admin (email) -->



<!-- Date: Fri Nov 22 10:02:33 2002 -->
<pre>Pasha 
 
Toolman runs a job every5min that checks collects data on the 
users logged on.   Reviewing the data for the cdfrep01 instance, 
the processes where never higher then 28.   We also have 
an alert that warns us if the process get to 70% of the 
allocation and we received not such alert for cdfrep01 ...ONLY the cdfofprd alert. 
 
Also, the sar info for fcdflnx1 reports expected no's. 
 
Nelly 
 
----- Original Message -----  
From: "Pasha Murat 630-840-8237/169G" &lt;murat@fnal.gov&gt; 
To: &lt;cdfdb-support@fnal.gov&gt; 
Cc: &lt;cdf_offline_operations@fnal.gov&gt;; &lt;menzemer@fnal.gov&gt; 
Sent: Friday, November 22, 2002 9:12 AM 
Subject: Re: Farm database load problem 
 
 
&gt; I just checked - 20 min ago we had the same problem with CDFPER01 - 192 connections 
&gt; opened - now we are back to 15. Do we have any information on where the 
&gt; connections to CDFPER01 were coming from? 
&gt;                                                         -thanks, Pasha 
&gt;  
&gt; Art Kreymer wrote: 
&gt; &gt; 
&gt; &gt; The cdfofprd database is being heavily loaded by the present farms processing. 
&gt; &gt; 
&gt; &gt; There are over 100 active connections, most from the farm, 
&gt; &gt; some of which have been connected and active for over an hour. 
&gt; &gt; Normally farm connections are active for about a minute, 
&gt; &gt; and may stay connected but idle for 5 minutes. 
&gt; &gt; But the farms seem to be running the standard 4.8.4a executeable. 
&gt; &gt; 
&gt; &gt; The root cause of this may involve a large number of active connections from 
&gt; &gt; user menzemer at nodes with names like c01-004-112,  at Karlsruhe. 
&gt; &gt; 
&gt; &gt; The cdfofprd database server has a load average of 40 right now, 
&gt; &gt; with all memory in use, and more than half the swap space in use. 
&gt; &gt; It is so badly loaded that even lightweight sqlplus queries 
&gt; &gt; are taking many minutes to perform. 
&gt; &gt; 
&gt; &gt; Please stop the memzemer jobs at Karlsruhe, while we investigate. 
&gt;</pre>
<!-- Author: nstanfield  -->

<!-- Date: Fri Nov 22 10:31:00 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Fri Nov 22 10:31:00 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Fri Nov 22 10:31:00 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Fri Nov 22 10:31:00 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Fri Nov 22 10:32:21 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Fri Nov 22 10:33:05 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->



<!-- Date: Fri Nov 22 10:35:32 2002 -->
<pre>refresh cdftrig_int with cdftrig_prd.. for donatella to test 
her clean up script... again.. she will then run in PRD  
 
 
The following tablespaces in cdfonint require coalescing: 
 
Tablespace Tablespace                     Coalescable 
    Number Name                               Extents 
---------- ------------------------------ ----------- 
         8 CDFTRIG_DATA                            29 
         9 CDFTRIG_IDX                             63 
        18 MLOG_DATA                               32 
beginning coalescing of cdfonint 
after the coalesce for cdfonint 
</pre>
<!-- Author: nstanfield  -->
<!-- Comment: Fri Nov 22 11;02;21 comment by...nstanfield/dtorretta  -->
<pre>Donatella  is ready to run her script that retires everything NOT
needed anymore for current physics tables.   We agreed to do 1
last test before she runs it in PRD next week.  I have refreshed
cdftrig_int with cdftrig_prd.   

We will also do a full export and check backups before she runs in PRD next week! </pre>


<!-- Date: Fri Nov 22 15:32:14 2002 -->
<pre>All 
 
We have just been informed that a bug was introduced 
when we installed the latest Enterprise Kernel on fcdflnx1, 
which houses the cdfrep01 database. 
 
This could lead to a denial of service and the 
sysadmin feels we need to install the patch they have 
provided. 
 
The machine will have to be rebooted.  We hope to 
be ready to do this at 4p.m. The interruption will 
be a few minutes and we will send email when the 
process is complete.    
 
We understand this is short notice, but it needs to 
be done. 
 
Nelly/Richard 
</pre>
<!-- Author: nstanfield -->


<!-- Date: Fri Nov 22 15:57:47 2002 -->
<pre>Subject  Designer 9 on b0dap15  
Bill / Jack / Dmitri 
 
I have installed Designer 9 on b0dap15...seems 
you guys all use this machine for Designer. 
 
Clean install of Designer and oracle_tnsnames, machine 
was recently wiped and updated to W2000..so I did not 
have to do an uninstall of 6. 
 
I will migrate applications: run, hardware, file catalog 
and calib..next week... I will let each of you know 
when I do. Once I migrate you have to use Designer 9, 
to access your design. 
 
Thanks 
Nelly 
 
  
  
</pre>
<!-- Author: nstanfield  -->
<!-- Comment: Fri Nov 22 16;15;57 comment by...ns -->
<pre>Richard has successfully installed the patch.
No further interruptions expected.</pre>


<!-- Date: Fri Nov 22 16:30:01 2002 -->
<pre>Idle Time was 0% on 21st Nov Thu during 23:20:00 to 23:58:00 It seems like os user 'safonov' issued lots of query against 
oracle data dictionary tables with some 'order by' clause.  
These queries were issued from node nucd03.fnal.gov 
User was trying to duplicate the query as we have 
for logintraces. Very strange. Why he was doing this ? 
 
'safonov' issued the queries against Oracle data dictionary 
tables around 22:58:22 on cdfofprd. And it was using tens of temporary space. 
Similary logic was tried by user 'zsenei'. 
Need to follow-up with this user why data was needed  from  
oracle dictionary tables. And what was sort order.  
And what's was the aim ? 
 
Also during this time cdfprod0 os account from fncdf108.fnal.gov issued the similar quieries. 
 
Another very very strange thing. Os user 'tqb' was trying to update the logoff time and cpu used in logintraces table. 
 
CDF_READER   tqb             nkek18.fnal.gov       140   60614 USERTEMP_01 
   SORT      UPDATE LOGINTRACE SET LOGOFF_T 
 
             IME=SYSDATE,CPU_USED=:b1 WHERE 
 
              SID = :b2  AND SERIAL# = :b3 
 
              AND LOGON_TIME = TO_DATE(:b4, 
 
             'mm/dd/yyyy hh24:mi:ss')  AND 
             LOGOFF_TIME IS NULL 
 
And Similar attempt was made by other users too. 
 
Update on logintraces for cpu_used columns and 
logofftime were  also issued by 
OS users : 'giolo' at 21:17 on 21st Nov, and  'safonov' 
on 23:47 on 21st Nov . 
 
This is an attempt to disrupt our internal audit of db 
usage. We need to follow up with these users promptly. 
 
</pre>
<!-- Author: akumar -->


<!-- Date: Fri Nov 22 17:34:44 2002 -->
<pre>&gt; runscript cdfofprd show_users | grep 27308 
27308           47   57128 CDF_READER   menzemer     c01-004-102          /grid/fzk.de/mounts/nfs/home/menzemer//j 
&gt; runscript cdfofprd show_users | grep 25546 
25546           66   22687 CDF_READER   menzemer     c01-001-104          /grid/fzk.de/mounts/nfs/home/menzemer//j 
&gt; runscript cdfofprd show_users | grep 4111 
4111            26   61607 CDF_READER   menzemer     c01-001-127          /grid/fzk.de/mounts/nfs/home/menzemer//j 
&gt; runscript cdfofprd show_users | grep 18621 
18621           39   13270 CDF_READER   menzemer     c01-004-122          /grid/fzk.de/mounts/nfs/home/menzemer//j 
 
SunOS fcdfora1 5.8 Generic_108528-02 sun4u    11/22/02 
 
00:00:00    %usr    %sys    %wio   %idle 
00:20:00      86      14       0       0 
00:40:01      86      14       0       0 
01:00:00      85      15       0       0 
01:20:00      86      14       0       0 
01:40:01      88      12       0       0 
02:00:00      88      12       0       0 
02:20:00      88      12       0       0 
02:40:01      87      12       0       0 
03:00:01      87      12       0       0 
03:20:00      88      12       0       0 
03:40:00      73       8       1      18 
04:00:00      62       6       1      31 
04:20:00      71       9       1      19 
04:40:00      68       7       1      25 
05:00:00      66       6       1      28 
05:20:00      62       6       2      30 
05:40:00      61       5       1      33 
06:00:00      55       5       1      39 
06:20:00      65       7       2      26 
06:40:00      30       6       1      63 
07:00:00      29       5       1      65 
07:20:00      17      15      49      19 
07:40:01      33      14      50       3 
08:00:01      20      21      55       4 
08:20:01      19      28      48       5 
08:40:02      77      17       3       3 
09:00:00      76       8       1      16 
09:20:00      16       6       3      75 
09:40:00      45       8       3      43 
10:00:00      43      10      11      37 
10:20:00      53      14      19      13 
10:40:00      50      13      17      20 
11:00:00      52      11       6      30 
11:20:00      49      16      21      13 
11:40:00      54      17      24       6 
12:00:00      57      17      20       6 
12:20:00      57      16      19       9 
12:40:00      56      10       2      31 
13:00:00      75      13       1      10 
13:20:00      67      13       3      17 
13:40:00      68      12       4      16 
14:00:00      74      13       6       6 
14:20:00      77      14       3       6 
14:40:01      82      13       2       4 
15:00:00      72      15       8       5 
15:20:00      80      15       3       2 
15:40:01      82      13       1       4 
16:00:00      79      13       1       7 
16:20:00      79      16       1       4 
16:40:00      79      14       1       6 
17:00:00      69      14       2      14 
17:20:00      76      16       1       7 
 
Average       64      12       8      16</pre>
<!-- Author: nstanfield -->


<!-- Date: Fri Nov 22 17:45:35 2002 -->
<pre>Hello Stephanie: 
 
There is a lot of FARM activity on fcdfora1 machine, 
cdfofprd instance.  This machine can barely 
handle that load, so we have set up a replica 
of the database on another machine fcdflnx1, 
instance name cdfrep01. 
 
Is it reasonable for you to change your jobs to 
use the replicated database? 
 
This other machine has a lot more cpu and 
can handle the load better.  This will also 
allow the Farms to continue uninterrupted. 
 
 
Thanks 
Nelly 
</pre>
<!-- Author: ns -->


<!-- Date: Fri Nov 22 17:48:46 2002 -->
<pre> 
Nelly and me just talked to Alexei Safonov. 
He was running TauPolar  program yesterday night.   
Just did the grep in the code of TauPolar for logintraces .  
No evidence found.  He is not even aware of any table called logintraces. 
 
Then he run the job again at 5:00PM. And  connections were  
coming and going out fine.We didn't see any evidence temp space being used by logintraces for update. 
 
Only yesterday night we saw couple of evidences when system was very busy. 
 
I also checked the permissions on logintraces.   
CDF_READER does not have any permissions to update logintraces tables.  So no RISK there. 
 
It looks like database trigger owned by TOOLMAN updates the logintraces table,but it shows the query and update on logintraces being done by DB user. 
But dictionary  shows  selects/updates being done on logintraces by db account  but actually not. 
 
So it is not the fault of any of users indicated in the e-mails. 
 
We will follow-up with ORACLE why updates on logintraces being shown by actual db users. 
 
Seems like we are fine. 
 
And I  am sorry for any confusion being caused. 
 
 
</pre>
<!-- Author: akumar -->

<!-- Date: Fri Nov 22 20:14:45 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Maximum Extents 
Segment CDFTRIG_PRD_REP.DATASETS_TRIGGER_PATHS in tablespace CDFTRIG_PRD_REP_DATA can only allocate 9 additional extents. Segment CDFPQCD.CDF2_FILE_LIVETIMES in tablespace USERDATA_01 can only allocate 9 additional extents. Segment CDFPQCD.CDF2_RUNSECTION_RANGES in tablespace USERDATA_01 can only allocate 3 additional extents. Segment CDFPBOT.CDF2_FILESETS in tablespace USERDATA_01 can only allocate 4 additional extents. Segment CDFPBOT.CDF2_FILES in tablespace USERDATA_01 can only allocate 3 additional extents. Segment CDFPBOT.CDF2_FILE_LIVETIMES in tablespace USERDATA_01 can only allocate 9 additional extents. Segment CDFPBOT.CDF2_RUNSECTION_RANGES in ... 
CRITICAL 
Nov 22 2002 08:13:39 PM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sat Nov 23 10:31:02 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sat Nov 23 10:31:02 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sat Nov 23 10:31:02 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sat Nov 23 10:31:02 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 05:47:19 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 05:46:34 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 05:52:32 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONPRD ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-12012: error on auto execute of job 3776 ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONPRD ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: ... 
WARNING 
Nov 24 2002 05:51:35 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 05:55:12 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-00600: internal error code, arguments: [12333], [4], [1], [0], [], [], [], []. 
CRITICAL 
Nov 24 2002 05:54:02 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 05:57:17 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-02068: following severe error from CDFONPRD ORA-03113: end-of-file on communication channel ORA-12008: error in snapshot refresh path ORA-12012: error on auto execute of job 3776 ORA-02068: following severe error from CDFONPRD ORA-03113: end-of-file on communication channel ORA-12008: error in snapshot refresh path ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 05:56:36 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 06:02:55 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-00600: internal error code, arguments: [15868], [27836048], [0], [37809], [], [], [], [] ORA-00604: error occurred at recursive SQL level 2 ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONPRD ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-12012: error on auto execute of job 3776 ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONP... 
CRITICAL 
Nov 24 2002 06:01:37 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 06:17:27 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 06:16:40 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 06:26:26 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Failed Jobs 
Event test "Failed Jobs" is cleared 
CLEARED 
Nov 24 2002 06:25:25 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 06:32:40 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONPRD ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-12012: error on auto execute of job 3776 ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONPRD ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: ... 
WARNING 
Nov 24 2002 06:31:43 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 06:47:29 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 06:46:46 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 07:08:12 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK. 
WARNING 
Nov 24 2002 07:06:50 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 07:10:17 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-00600: internal error code, arguments: [4400], [48], [], [], [], [], [], [] ORA-00600: internal error code, arguments: [17182], [2231134152], [], [], [], [], [], [] ORA-00600: internal error code, arguments: [4400], [48], [], [], [], [], [], []. 
CRITICAL 
Nov 24 2002 07:09:07 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 07:12:51 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONPRD ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-12012: error on auto execute of job 3776 ORA-03106: fatal two-task communication protocol error ORA-02063: preceding line from CDFONPRD ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH"... 
WARNING 
Nov 24 2002 07:11:51 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 07:26:52 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Failed Jobs 
Event test "Failed Jobs" is cleared 
CLEARED 
Nov 24 2002 07:25:26 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 07:37:54 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 07:36:56 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 07:53:02 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 07:52:00 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 08:08:16 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 08:07:02 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 08:17:41 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 08:17:04 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 08:28:03 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Failed Jobs 
Event test "Failed Jobs" is cleared 
CLEARED 
Nov 24 2002 08:25:27 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 08:34:16 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 08:32:07 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 08:48:46 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 08:47:10 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 09:03:09 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 09:02:13 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 09:19:03 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 09:17:16 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 09:26:06 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Failed Jobs 
Event test "Failed Jobs" is cleared 
CLEARED 
Nov 24 2002 09:25:29 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 09:33:22 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 09:32:19 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 09:48:51 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 09:47:22 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:03:21 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 10:02:25 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:19:05 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 10:17:28 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:27:22 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Failed Jobs 
Event test "Failed Jobs" is cleared 
CLEARED 
Nov 24 2002 10:25:29 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:31:31 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:31:31 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:31:31 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:31:31 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:34:26 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 10:32:31 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 10:48:56 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 10:47:34 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 11:03:27 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 11:02:37 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 11:18:29 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 11:17:40 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 11:26:14 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Failed Jobs 
Event test "Failed Jobs" is cleared 
CLEARED 
Nov 24 2002 11:25:30 AM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 11:38:40 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Alert 
The following errors are in the ALERT file : ORA-1632: max # extents 20 reached in index RUNDB_PRD_REP.SC_PK  ORA-12012: error on auto execute of job 3776 ORA-12008: error in snapshot refresh path ORA-01632: max # extents (20) reached in index RUNDB_PRD_REP.SC_PK ORA-06512: at "SYS.DBMS_SNAPSHOT", line 617 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 674 ORA-06512: at "SYS.DBMS_IREFRESH", line 577 ORA-06512: at "SYS.DBMS_REFRESH", line 211 ORA-06512: at line 1. 
WARNING 
Nov 24 2002 11:37:44 AM 


</pre>
<!-- Author: oem-admin (email) -->



<!-- Date: Sun Nov 24 11:57:41 2002 -->
<pre>on cdfofprd .... 
 
ALTER INDEX "RUNDB_PRD_REP"."SC_PK" 
    STORAGE ( MAXEXTENTS 40)</pre>
<!-- Author: ns  -->

<!-- Date: Sun Nov 24 12:26:23 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Failed Jobs 
Event test "Failed Jobs" is cleared 
CLEARED 
Nov 24 2002 12:25:31 PM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Sun Nov 24 14:15:55 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 

Maximum Extents 
Segment CDFTRIG_PRD_REP.DATASETS_TRIGGER_PATHS in tablespace CDFTRIG_PRD_REP_DATA can only allocate 9 additional extents. Segment CDFPQCD.CDF2_FILE_LIVETIMES in tablespace USERDATA_01 can only allocate 9 additional extents. Segment CDFPQCD.CDF2_RUNSECTION_RANGES in tablespace USERDATA_01 can only allocate 3 additional extents. Segment CDFPBOT.CDF2_FILESETS in tablespace USERDATA_01 can only allocate 4 additional extents. Segment CDFPBOT.CDF2_FILES in tablespace USERDATA_01 can only allocate 3 additional extents. Segment CDFPBOT.CDF2_FILE_LIVETIMES in tablespace USERDATA_01 can only allocate 9 additional extents. Segment CDFPBOT.CDF2_RUNSECTION_RANGES in ... 
CRITICAL 
Nov 24 2002 02:13:39 PM 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Mon Nov 25 10:31:28 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Mon Nov 25 10:31:28 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Mon Nov 25 10:31:28 2002 -->
<pre>
Target Name : cdfofprd.fnal.gov 
Target Type : Database 
Node Name : fcdfora1.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->


<!-- Date: Mon Nov 25 10:31:28 2002 -->
<pre>
Target Name : cdfonprd.fnal.gov 
Target Type : Database 
Node Name : b0dau35.fnal.gov 


</pre>
<!-- Author: oem-admin (email) -->