Oracle JCA Adapters are designed to process large payloads but the BPEL engine consumes huge memory when processing large payloads due to XML conversions which can cause well known OutOfMemory(OOM) and jeopardize the whole system.Setting the payload threshold prevents any data burst from the downstream systems ensuring the JCA Adapters process only payloads that are less than the threshold limit and reject others.
Capping the payload size according to the capacity of the infrastructure in place is always preferred since the servers will take time to recover from OOM error by garbage collection which is a "Stop the World" activity and no requests will be processed by the server during the time.
The threshold for maximum request size for any exposed service can be set by using this attribute.The value can be set through the EM console by navigating to the service endpoint -> Service/Reference Properties ->Exposed Service. A default value of -1 represents unlimited which can be set in units of bytes,KB,MB or GB to a suitable value that your infrastructure can handle.
This can be restricted at the DMZ level without the requests reaching the web logic servers if the architecture includes any of the below components.
Capping the payload size according to the capacity of the infrastructure in place is always preferred since the servers will take time to recover from OOM error by garbage collection which is a "Stop the World" activity and no requests will be processed by the server during the time.
Maximum Request size for services
This can be restricted at the DMZ level without the requests reaching the web logic servers if the architecture includes any of the below components.
- Oracle API Gateway
- Web Server that can restrict request size
- Load balancer like F5 BIG IP
payloadSizeThreshold for adapters
Setting the payload threshold ensures that Oracle JCA Adapters rejects payloads greater than the threshold limit. In case of file and FTP adapters when the native size of the payload is not available and if the specific adapter does not use the native translation library, you cannot enforce the payload size threshold limit. For example, in case of xml-debatching, where the Oracle File and FTP Adapters pass a chunk of file content and the actual native size is not known, payload size threshold limit cannot be used.
To set the global property for capping payload size login to the EM console -> soa-infra -> Administration -> System MBean Browser - > adapter and set the value for DefaultPayloadSizeThreshold . As per the documentation DefaultPayloadSizeThreshold works only for the inbound adapters.There is no attribute as payloadSizeThreshold which seems to be a bug and DefaultPayloadSizeThreshold works for both inbound and outbound operations.
This threshold can be increased/decreased at adapter service level by overriding the values in composite.xml which takes precedence over the global settings.
<reference name="insert" ui:wsdlLocation="insert.wsdl">
<interface.wsdl interface="http://xmlns.oracle.com/pcbpel/adapter/db/Application3/One2ManyJoining/insert#wsdl.interface(insert_ptt)"/>
<binding.jca config="insert_db.jca"/>
<property name="payloadSizeThreshold" type="xs:string" many="false" override="may">100</property> </reference>
Below are some of the options when it is required to process huge data using database adapter.
- MaxRaiseSize and MaxTransactionSize can be used while using database polling to restrict the number of records returned from the adapter.
- Setting the audit level to minimal can enhance the processing of large data with a drawback of payload not getting saved to the database
- When processing large payload using XSLT set "streamResultToTempFile" to yes to avoid OutOfMemory error.But assign activities always perform better.
- Parking Lot pattern can be used to throttle the messages processed.See Throttling in SOA Suite via Parking Lot Pattern at http://www.ateam-oracle.com/throttling-in-soa-suite-via-parking-lot-pattern/ for how to implement in SOA.
Other adapters like AQ,MQ,File,FTP etc... have their own parameters that can be tweaked to control the payload size when used for inbound operations.
