Saturday, September 21, 2013

ORA-00942: table or view does not exist ORA-06512: at "DEV_MDS.MDS_INTERNAL_SHREDDED"

The composite which is working fine may fail with the below errors.This issue was observed in SOA Suite 11.1.1.7.0 and it may error out when you try to undeploy composite using the WLST script.Also it may prevent you from deploying any new composite into the SOA domain.


MDS-00054: The File to be Loaded oramds:/deployed-composites/default/Test_rev1.0/SCA-INF/classes/META-INF/adf-config.xml Does Not Exist

ORA-00942: table or view does not exist ORA-06512: at "DEV_MDS.MDS_INTERNAL_SHREDDED", line 37 ORA-06512: at "DEV_MDS.MDS_INTERNAL_SHREDDED", line 664 ORA-06512: at line 1


This error happens due to the MDS corruption when there are multiple schemas with identical objects.In our case we had two installations of the SOA pointing to the same database with different prefix.

Connect to the MDS database as sys user and run the below script.

alter system flush shared_pool;


There is a database patch available to prevent future corruption(Doc ID 9577583.8)

Thursday, September 5, 2013

Oracle SOA purge instances

Deleting Large Numbers of Instances with the Purge Scripts

Deleting thousands of instances with the Delete With Options button on the Instances page of a SOA composite application in Oracle Enterprise Manager Fusion Middleware Control takes time and may result in a transaction timeout.At times it brings down the SOA server with heap space error.Instead we can use the purge scripts for deleting instances.

Get total number of instances

The following query can be used to get the total number of instances stored in the database.

 SELECT  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.CUBE_INSTANCE) AS CUBE_INST,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.COMPOSITE_INSTANCE) AS COMP_INST,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.COMPONENT_INSTANCE) AS COMPNT_INST,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.COMPOSITE_INSTANCE_FAULT) AS COMPST_INST_FLT,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.REFERENCE_INSTANCE) AS REF_INST,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.CUBE_SCOPE) AS CUBE_SCP,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.COMPOSITE_SENSOR_VALUE) AS COMP_SEN_VAL,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.XML_DOCUMENT) AS XML_DOC,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.XML_DOCUMENT_REF) AS XML_DOC_REF,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.AUDIT_TRAIL) AS AUDIT_TRAIL,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.AUDIT_COUNTER) AS AUDIT_CNTR,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.AUDIT_DETAILS) AS AUDIT_DET,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.INSTANCE_PAYLOAD) AS INST_PYLD,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.HEADERS_PROPERTIES) AS HDRS_PROPS,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.WFTASK) AS WFTASK,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.WI_FAULT) AS WI_FLT,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.DLV_MESSAGE) AS DLV_MSSG,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.DLV_SUBSCRIPTION) AS DLV_SUBC,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.DOCUMENT_DLV_MSG_REF) AS DOC_DLV_MSG_REF,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.DOCUMENT_CI_REF) AS DOC_CI_REF,  
 (SELECT COUNT(*) FROM DEV_SOAINFRA.WORK_ITEM) AS WRK_ITEM  
 FROM DUAL;  

To know  the available table space  below query can be used.

 SELECT * FROM (SELECT C.TABLESPACE_NAME, ROUND(A.BYTES/1048576,2) MB_ALLOCATED,ROUND(B.BYTES/1048576,2)  
 MB_FREE,ROUND((A.BYTES-B.BYTES)/1048576,2) MB_USED, ROUND(B.BYTES/A.BYTES  
 * 100,2) TOT_PCT_FREE, ROUND((A.BYTES-B.BYTES)/A.BYTES,2) * 100 TOT_PCT_USED FROM (SELECT TABLESPACE_NAME, SUM(A.BYTES) BYTES FROM  
 SYS.DBA_DATA_FILES A GROUP BY TABLESPACE_NAME) A, (SELECT A.TABLESPACE_NAME,  
 NVL(SUM(B.BYTES),0) BYTES FROM SYS.DBA_DATA_FILES A, SYS.DBA_FREE_SPACE B WHERE A.TABLESPACE_NAME = B.TABLESPACE_NAME (+)  
 AND A.FILE_ID = B.FILE_ID (+) GROUP BY A.TABLESPACE_NAME) B,  
 SYS.DBA_TABLESPACES C WHERE A.TABLESPACE_NAME = B.TABLESPACE_NAME(+) AND  
 A.TABLESPACE_NAME = C.TABLESPACE_NAME) WHERE TOT_PCT_USED >=0 AND  
 TABLESPACE_NAME='DEV_SOAINFRA' ORDER BY TABLESPACE_NAME;  

Running the Purge Scripts

This section describes how to execute the purge scripts.
To execute the purge scripts:

This section describes how to execute the purge scripts.
To execute the purge scripts:

1.In SQL*Plus, connect to the database AS SYSDBA:

CONNECT SYS AS SYSDBA

2.Execute the following SQL commands:

GRANT EXECUTE ON DBMS_LOCK TO USER;
GRANT CREATE ANY JOB TO USER;

where USER is the soainfra account to execute the scripts. These privileges are required to run the scripts.

3.Load the purge scripts by running the main purge script in the MW_HOME/SOA_ORACLE_HOME/rcu/integration/soainfra/sql/soa_purge directory.
For a parallel purge, the debug logs from the jobs spawned by a parallel purge are logged into files created in the directory named SOA_PURGE_DIR. This directory must be accessible to the Oracle database.

4.Create SOA_PURGE_DIR and grant write permissions to the soainfra user.
mkdir -p /tmp/purgelog

5.CREATE OR REPLACE DIRECTORY SOA_PURGE_DIR AS 'SERVER_DIRECTORY'
where SERVER_DIRECTORY is the name of the directory to create (for example, '/tmp/purgelog/'). Note the required single quotes around the directory path.


6.If you want to execute the scripts in debug mode, run common/debug_on.sql and set serverout to on in SQL*Plus. This step is optional.
SET SERVEROUTPUT ON
The logs from the spawned jobs are logged into the directory created in Step 4 (separate files per job). The rest of the logs are displayed on stdout(or the spool file, if configured).

There are two options for purging:

  • Looped purge
  • Parallel purge


Execute the purge scripts as shown below. Examples are provided for both options.
For looped purge:

 DECLARE  
   MAX_CREATION_DATE timestamp;  
   MIN_CREATION_DATE timestamp;  
   batch_size integer;  
   max_runtime integer;  
   retention_period timestamp;  
  BEGIN  
   MIN_CREATION_DATE := to_timestamp('2012-01-01','YYYY-MM-DD');  
   MAX_CREATION_DATE := to_timestamp('2012-01-31','YYYY-MM-DD');  
   max_runtime := 60;  
   retention_period := to_timestamp('2012-01-31','YYYY-MM-DD');  
   batch_size := 10000;  
    soa.delete_instances(  
    min_creation_date => MIN_CREATION_DATE,  
    max_creation_date => MAX_CREATION_DATE,  
    batch_size => batch_size,  
    max_runtime => max_runtime,  
    retention_period => retention_period,  
    purge_partitioned_component => false);  
  END;  
  /  

For parallel purge:


 DECLARE  
   max_creation_date timestamp;  
   min_creation_date timestamp;  
   retention_period timestamp;  
  BEGIN  
   min_creation_date := to_timestamp('2012-01-01','YYYY-MM-DD');  
   max_creation_date := to_timestamp('2012-01-31','YYYY-MM-DD');  
   retention_period := to_timestamp('2012-01-31','YYYY-MM-DD');  
   soa.delete_instances_in_parallel(  
    min_creation_date => min_creation_date,  
    max_creation_date => max_creation_date,  
    batch_size => 10000,  
    max_runtime => 60,  
    retention_period => retention_period,  
    DOP => 3,  
    max_count => 1000000,  
    purge_partitioned_component => false);  
  END;  

Monitoring the purge jobs

Below script can be used to get the status of the jobs.

SELECT * FROM dba_scheduler_jobs where job_name like '%SOA_PURGE%'

Reclaim Disk Space from Dehydration Table

For some Oracle SOA Suite environments, the purge scripts may not always remove a sufficient number of composite instances to reclaim database space. This can lead to performance issues.To reclaim the space following scripts can be used.


 ALTER SESSION SET CURRENT_SCHEMA=DEV_SOAINFRA;  
 
 ALTER TABLE MEDIATOR_CASE_INSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_CASE_INSTANCE SHRINK SPACE;  
 ALTER TABLE MEDIATOR_CASE_INSTANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_AUDIT_DOCUMENT ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_AUDIT_DOCUMENT SHRINK SPACE;  
 ALTER TABLE MEDIATOR_AUDIT_DOCUMENT DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_CALLBACK ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_CALLBACK SHRINK SPACE;  
 ALTER TABLE MEDIATOR_CALLBACK DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_GROUP_STATUS ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_GROUP_STATUS SHRINK SPACE;  
 ALTER TABLE MEDIATOR_GROUP_STATUS DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_PAYLOAD ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_PAYLOAD SHRINK SPACE;  
 ALTER TABLE MEDIATOR_PAYLOAD DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_DEFERRED_MESSAGE ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_DEFERRED_MESSAGE SHRINK SPACE;  
 ALTER TABLE MEDIATOR_DEFERRED_MESSAGE DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_RESEQUENCER_MESSAGE ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_RESEQUENCER_MESSAGE SHRINK SPACE;  
 ALTER TABLE MEDIATOR_RESEQUENCER_MESSAGE DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_CASE_DETAIL ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_CASE_DETAIL SHRINK SPACE;  
 ALTER TABLE MEDIATOR_CASE_DETAIL DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_CORRELATION ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_CORRELATION SHRINK SPACE;  
 ALTER TABLE MEDIATOR_CORRELATION DISABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_INSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE MEDIATOR_INSTANCE SHRINK SPACE;  
 ALTER TABLE MEDIATOR_INSTANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE HEADERS_PROPERTIES ENABLE ROW MOVEMENT;  
 ALTER TABLE HEADERS_PROPERTIES SHRINK SPACE;  
 ALTER TABLE HEADERS_PROPERTIES DISABLE ROW MOVEMENT;  
 ALTER TABLE AG_INSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE AG_INSTANCE SHRINK SPACE;  
 ALTER TABLE AG_INSTANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE AUDIT_COUNTER ENABLE ROW MOVEMENT;  
 ALTER TABLE AUDIT_COUNTER SHRINK SPACE;  
 ALTER TABLE AUDIT_COUNTER DISABLE ROW MOVEMENT;  
 ALTER TABLE AUDIT_TRAIL ENABLE ROW MOVEMENT;  
 ALTER TABLE AUDIT_TRAIL SHRINK SPACE;  
 ALTER TABLE AUDIT_TRAIL DISABLE ROW MOVEMENT;  
 ALTER TABLE AUDIT_DETAILS ENABLE ROW MOVEMENT;  
 ALTER TABLE AUDIT_DETAILS SHRINK SPACE;  
 ALTER TABLE AUDIT_DETAILS DISABLE ROW MOVEMENT;  
 ALTER TABLE CI_INDEXES ENABLE ROW MOVEMENT;  
 ALTER TABLE CI_INDEXES SHRINK SPACE;  
 ALTER TABLE CI_INDEXES DISABLE ROW MOVEMENT;  
 ALTER TABLE WORK_ITEM ENABLE ROW MOVEMENT;  
 ALTER TABLE WORK_ITEM SHRINK SPACE;  
 ALTER TABLE WORK_ITEM DISABLE ROW MOVEMENT;  
 ALTER TABLE WI_FAULT ENABLE ROW MOVEMENT;  
 ALTER TABLE WI_FAULT SHRINK SPACE;  
 ALTER TABLE WI_FAULT DISABLE ROW MOVEMENT;  
 ALTER TABLE XML_DOCUMENT_REF ENABLE ROW MOVEMENT;  
 ALTER TABLE XML_DOCUMENT_REF SHRINK SPACE;  
 ALTER TABLE XML_DOCUMENT_REF DISABLE ROW MOVEMENT;  
 ALTER TABLE DOCUMENT_DLV_MSG_REF ENABLE ROW MOVEMENT;  
 ALTER TABLE DOCUMENT_DLV_MSG_REF SHRINK SPACE;  
 ALTER TABLE DOCUMENT_DLV_MSG_REF DISABLE ROW MOVEMENT;  
 ALTER TABLE DOCUMENT_CI_REF ENABLE ROW MOVEMENT;  
 ALTER TABLE DOCUMENT_CI_REF SHRINK SPACE;  
 ALTER TABLE DOCUMENT_CI_REF DISABLE ROW MOVEMENT;  
 ALTER TABLE DLV_SUBSCRIPTION ENABLE ROW MOVEMENT;  
 ALTER TABLE DLV_SUBSCRIPTION SHRINK SPACE;  
 ALTER TABLE DLV_SUBSCRIPTION DISABLE ROW MOVEMENT;  
 ALTER TABLE DLV_MESSAGE ENABLE ROW MOVEMENT;  
 ALTER TABLE DLV_MESSAGE SHRINK SPACE;  
 ALTER TABLE DLV_MESSAGE DISABLE ROW MOVEMENT;  
 ALTER TABLE REJECTED_MSG_NATIVE_PAYLOAD ENABLE ROW MOVEMENT;  
 ALTER TABLE REJECTED_MSG_NATIVE_PAYLOAD SHRINK SPACE;  
 ALTER TABLE REJECTED_MSG_NATIVE_PAYLOAD DISABLE ROW MOVEMENT;  
 ALTER TABLE INSTANCE_PAYLOAD ENABLE ROW MOVEMENT;  
 ALTER TABLE INSTANCE_PAYLOAD SHRINK SPACE;  
 ALTER TABLE INSTANCE_PAYLOAD DISABLE ROW MOVEMENT;  
 ALTER TABLE TEST_DETAILS ENABLE ROW MOVEMENT;  
 ALTER TABLE TEST_DETAILS SHRINK SPACE;  
 ALTER TABLE TEST_DETAILS DISABLE ROW MOVEMENT;  
 ALTER TABLE CUBE_SCOPE ENABLE ROW MOVEMENT;  
 ALTER TABLE CUBE_SCOPE SHRINK SPACE;  
 ALTER TABLE CUBE_SCOPE DISABLE ROW MOVEMENT;  
 ALTER TABLE CUBE_INSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE CUBE_INSTANCE SHRINK SPACE;  
 ALTER TABLE CUBE_INSTANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE BPM_AUDIT_QUERY ENABLE ROW MOVEMENT;  
 ALTER TABLE BPM_AUDIT_QUERY SHRINK SPACE;  
 ALTER TABLE BPM_AUDIT_QUERY DISABLE ROW MOVEMENT;  
 ALTER TABLE BPM_MEASUREMENT_ACTIONS ENABLE ROW MOVEMENT;  
 ALTER TABLE BPM_MEASUREMENT_ACTIONS SHRINK SPACE;  
 ALTER TABLE BPM_MEASUREMENT_ACTIONS DISABLE ROW MOVEMENT;  
 ALTER TABLE BPM_MEASUREMENT_ACTION_EXCEPS ENABLE ROW MOVEMENT;  
 ALTER TABLE BPM_MEASUREMENT_ACTION_EXCEPS SHRINK SPACE;  
 ALTER TABLE BPM_MEASUREMENT_ACTION_EXCEPS DISABLE ROW MOVEMENT;  
 ALTER TABLE BPM_CUBE_AUDITINSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE BPM_CUBE_AUDITINSTANCE SHRINK SPACE;  
 ALTER TABLE BPM_CUBE_AUDITINSTANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE BPM_CUBE_TASKPERFORMANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE BPM_CUBE_TASKPERFORMANCE SHRINK SPACE;  
 ALTER TABLE BPM_CUBE_TASKPERFORMANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE BPM_CUBE_PROCESSPERFORMANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE BPM_CUBE_PROCESSPERFORMANCE SHRINK SPACE;  
 ALTER TABLE BPM_CUBE_PROCESSPERFORMANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASK_TL ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASK_TL SHRINK SPACE;  
 ALTER TABLE WFTASK_TL DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKHISTORY ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKHISTORY SHRINK SPACE;  
 ALTER TABLE WFTASKHISTORY DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKHISTORY_TL ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKHISTORY_TL SHRINK SPACE;  
 ALTER TABLE WFTASKHISTORY_TL DISABLE ROW MOVEMENT;  
 ALTER TABLE WFCOMMENTS ENABLE ROW MOVEMENT;  
 ALTER TABLE WFCOMMENTS SHRINK SPACE;  
 ALTER TABLE WFCOMMENTS DISABLE ROW MOVEMENT;  
 ALTER TABLE WFMESSAGEATTRIBUTE ENABLE ROW MOVEMENT;  
 ALTER TABLE WFMESSAGEATTRIBUTE SHRINK SPACE;  
 ALTER TABLE WFMESSAGEATTRIBUTE DISABLE ROW MOVEMENT;  
 ALTER TABLE WFATTACHMENT ENABLE ROW MOVEMENT;  
 ALTER TABLE WFATTACHMENT SHRINK SPACE;  
 ALTER TABLE WFATTACHMENT DISABLE ROW MOVEMENT;  
 ALTER TABLE WFASSIGNEE ENABLE ROW MOVEMENT;  
 ALTER TABLE WFASSIGNEE SHRINK SPACE;  
 ALTER TABLE WFASSIGNEE DISABLE ROW MOVEMENT;  
 ALTER TABLE WFREVIEWER ENABLE ROW MOVEMENT;  
 ALTER TABLE WFREVIEWER SHRINK SPACE;  
 ALTER TABLE WFREVIEWER DISABLE ROW MOVEMENT;  
 ALTER TABLE WFCOLLECTIONTARGET ENABLE ROW MOVEMENT;  
 ALTER TABLE WFCOLLECTIONTARGET SHRINK SPACE;  
 ALTER TABLE WFCOLLECTIONTARGET DISABLE ROW MOVEMENT;  
 ALTER TABLE WFROUTINGSLIP ENABLE ROW MOVEMENT;  
 ALTER TABLE WFROUTINGSLIP SHRINK SPACE;  
 ALTER TABLE WFROUTINGSLIP DISABLE ROW MOVEMENT;  
 ALTER TABLE WFNOTIFICATION ENABLE ROW MOVEMENT;  
 ALTER TABLE WFNOTIFICATION SHRINK SPACE;  
 ALTER TABLE WFNOTIFICATION DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKTIMER ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKTIMER SHRINK SPACE;  
 ALTER TABLE WFTASKTIMER DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKERROR ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKERROR SHRINK SPACE;  
 ALTER TABLE WFTASKERROR DISABLE ROW MOVEMENT;  
 ALTER TABLE WFHEADERPROPS ENABLE ROW MOVEMENT;  
 ALTER TABLE WFHEADERPROPS SHRINK SPACE;  
 ALTER TABLE WFHEADERPROPS DISABLE ROW MOVEMENT;  
 ALTER TABLE WFEVIDENCE ENABLE ROW MOVEMENT;  
 ALTER TABLE WFEVIDENCE SHRINK SPACE;  
 ALTER TABLE WFEVIDENCE DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKASSIGNMENTSTATISTIC ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKASSIGNMENTSTATISTIC SHRINK SPACE;  
 ALTER TABLE WFTASKASSIGNMENTSTATISTIC DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKAGGREGATION ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASKAGGREGATION SHRINK SPACE;  
 ALTER TABLE WFTASKAGGREGATION DISABLE ROW MOVEMENT;  
 ALTER TABLE WFTASK ENABLE ROW MOVEMENT;  
 ALTER TABLE WFTASK SHRINK SPACE;  
 ALTER TABLE WFTASK DISABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_SENSOR_VALUE ENABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_SENSOR_VALUE SHRINK SPACE;  
 ALTER TABLE COMPOSITE_SENSOR_VALUE DISABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_INSTANCE_ASSOC ENABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_INSTANCE_ASSOC SHRINK SPACE;  
 ALTER TABLE COMPOSITE_INSTANCE_ASSOC DISABLE ROW MOVEMENT;  
 ALTER TABLE ATTACHMENT ENABLE ROW MOVEMENT;  
 ALTER TABLE ATTACHMENT SHRINK SPACE;  
 ALTER TABLE ATTACHMENT DISABLE ROW MOVEMENT;  
 ALTER TABLE ATTACHMENT_REF ENABLE ROW MOVEMENT;  
 ALTER TABLE ATTACHMENT_REF SHRINK SPACE;  
 ALTER TABLE ATTACHMENT_REF DISABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_INSTANCE_FAULT ENABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_INSTANCE_FAULT SHRINK SPACE;  
 ALTER TABLE COMPOSITE_INSTANCE_FAULT DISABLE ROW MOVEMENT;  
 ALTER TABLE REFERENCE_INSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE REFERENCE_INSTANCE SHRINK SPACE;  
 ALTER TABLE REFERENCE_INSTANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE COMPONENT_INSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE COMPONENT_INSTANCE SHRINK SPACE;  
 ALTER TABLE COMPONENT_INSTANCE DISABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_INSTANCE ENABLE ROW MOVEMENT;  
 ALTER TABLE COMPOSITE_INSTANCE SHRINK SPACE;  
 ALTER TABLE COMPOSITE_INSTANCE DISABLE ROW MOVEMENT;  
 -- -----------------------------------------------------------------------------  
 -- Modify LOB Segments  
 -- -----------------------------------------------------------------------------  
 ALTER TABLE AUDIT_DETAILS MODIFY LOB (BIN) (SHRINK SPACE);  
 ALTER TABLE COMPOSITE_INSTANCE_FAULT MODIFY LOB (ERROR_MESSAGE) (SHRINK SPACE);  
 ALTER TABLE COMPOSITE_INSTANCE_FAULT MODIFY LOB (STACK_TRACE) (SHRINK SPACE);  
 ALTER TABLE CUBE_SCOPE MODIFY LOB (SCOPE_BIN) (SHRINK SPACE);  
 ALTER TABLE REFERENCE_INSTANCE MODIFY LOB (ERROR_MESSAGE) (SHRINK SPACE);  
 ALTER TABLE REFERENCE_INSTANCE MODIFY LOB (STACK_TRACE) (SHRINK SPACE);  
 ALTER TABLE TEST_DEFINITIONS MODIFY LOB (DEFINITION) (SHRINK SPACE);  
 ALTER TABLE WI_FAULT MODIFY LOB (MESSAGE) (SHRINK SPACE);  
 ALTER TABLE XML_DOCUMENT MODIFY LOB (DOCUMENT) (SHRINK SPACE);  
 -- -----------------------------------------------------------------------------  
 -- Rebuild Indexes  
 -- -----------------------------------------------------------------------------  
 ALTER INDEX AD_PK REBUILD ONLINE;  
 ALTER INDEX AT_PK REBUILD ONLINE;  
 ALTER INDEX CI_CREATION_DATE REBUILD ONLINE;  
 ALTER INDEX CI_CUSTOM3 REBUILD ONLINE;  
 ALTER INDEX CI_ECID REBUILD ONLINE;  
 ALTER INDEX CI_NAME_REV_STATE REBUILD ONLINE;  
 ALTER INDEX CI_PK REBUILD ONLINE;  
 ALTER INDEX COMPOSITE_INSTANCE_CIDN REBUILD ONLINE;  
 ALTER INDEX COMPOSITE_INSTANCE_CO_ID REBUILD ONLINE;  
 ALTER INDEX COMPOSITE_INSTANCE_CREATED REBUILD ONLINE;  
 ALTER INDEX COMPOSITE_INSTANCE_ECID REBUILD ONLINE;  
 ALTER INDEX COMPOSITE_INSTANCE_ID REBUILD ONLINE;  
 ALTER INDEX COMPOSITE_INSTANCE_STATE REBUILD ONLINE;  
 ALTER INDEX CS_PK REBUILD ONLINE;  
 ALTER INDEX DM_CONVERSATION REBUILD ONLINE;  
 ALTER INDEX DLV_MESSAGE_PK REBUILD ONLINE;  
 ALTER INDEX DOC_DLV_MSG_GUID_INDEX REBUILD ONLINE;  
 ALTER INDEX DOC_STORE_PK REBUILD ONLINE;  
 ALTER INDEX DS_CONVERSATION REBUILD ONLINE;  
 ALTER INDEX DS_CONV_STATE REBUILD ONLINE;  
 ALTER INDEX DS_FK REBUILD ONLINE;  
 ALTER INDEX DLV_SUBSCRIPTION_PK REBUILD ONLINE;  
 ALTER INDEX HEADERS_PROPERTIES_PK REBUILD ONLINE;  
 ALTER INDEX INSTANCE_PAYLOAD_KEY REBUILD ONLINE;  
 ALTER INDEX REFERENCE_INSTANCE_CDN_STATE REBUILD ONLINE;  
 ALTER INDEX REFERENCE_INSTANCE_CO_ID REBUILD ONLINE;  
 ALTER INDEX REFERENCE_INSTANCE_ECID REBUILD ONLINE;  
 ALTER INDEX REFERENCE_INSTANCE_ID REBUILD ONLINE;  
 ALTER INDEX REFERENCE_INSTANCE_STATE REBUILD ONLINE;  
 ALTER INDEX REFERENCE_INSTANCE_TIME_CDN REBUILD ONLINE;  
 ALTER INDEX DM_RECEIVE_DATE REBUILD ONLINE;  
 ALTER INDEX WF_CRDATE_CIKEY REBUILD ONLINE;  
 ALTER INDEX WF_CRDATE_TYPE REBUILD ONLINE;  
 ALTER INDEX WF_FK2 REBUILD ONLINE;  
 ALTER INDEX WI_FAULT_PK REBUILD ONLINE;  
 ALTER INDEX WI_EXPIRED REBUILD ONLINE;  
 ALTER INDEX WI_STATE_KEY REBUILD ONLINE;  
 ALTER INDEX WORK_ITEM_PK REBUILD ONLINE;  
 ALTER INDEX WI_EXPIRED REBUILD ONLINE;  
 ALTER INDEX XML_DOCUMENT_REF_PK REBUILD ONLINE;  


Tuesday, September 3, 2013

Server failed to get a connection to the database in the past 30 seconds for lease renewal. Server will shut itself down.

If you are using a database to manage leasing information , the server renews its "lease" by updating the timestamp in the lease table.

If a migratable server fails to reach the lease table and renew its lease before    the lease expires, it terminates as quickly as possible using a Java System.exit command and you can see a error trace as below:

<Sep 3, 2013 8:22:30 AM GST> <Warning> <Cluster> <BEA-000147> <Server "wls_soa2" failed to renew lease in the database.> 
<Sep 3, 2013 8:22:30 AM GST> <Error> <Cluster> <BEA-000150> <Server failed to get a connection to the database in the past 30 seconds for lease renewal. Server will shut itself down.> 
<Sep 3, 2013 8:22:30 AM GST> <Critical> <Health> <BEA-310006> <Critical Subsystem ServerMigration has failed. Setting server state to FAILED. 
Reason: Serverwls_soa2 failed to renew lease in the database> 
<Sep 3, 2013 8:22:30 AM GST> <Critical> <WebLogicServer> <BEA-000385> <Server health failed. Reason: health of critical service 'ServerMigration' failed> 
<Sep 3, 2013 8:22:30 AM GST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED> 
<Sep 3, 2013 8:22:30 AM GST> <Error> <> <BEA-000000>


To resolve  this issue please ensure that same time is set on all backend RAC database  nodes to which leasing datasource is connecting to.

This can also happen due to slow database connections and the timeout can be modified using the below properties.


By default a migratable server renews its lease every 30,000 milliseconds—the
product of two configurable ServerMBean properties:
– HealthCheckIntervalMillis, which by default is 10,000.
– HealthCheckPeriodsUntilFencing, which by default is 3.

Consensus leasing can be a option since it doesn't have the database dependency.