Error: Could not find or load main classorg.apache.hadoop.hdfs.server.namenode.NameNode

Error: Could not find or load main class org.apache.hadoop.hdfs.server.namenode.NameNode

This error "Error: Could not find or load main class org.apache.hadoop.hdfs.server.namenode.NameNode" can be seen When you try to execute the below command.

$ bin/hdfs namenode -format

The most common reason for this is the HADOOP_PREFIX environment variable is not set or to the correct path atleast.To set the path to the current session execute the following or set it in the profile permanently:

$ export HADOOP_PREFIX=/path_to_hadoop_location

For example,

$ export HADOOP_PREFIX= /u01/bigdata/hadoop

Once the value is set and running the command again the following start up messages should be seen.

-bash-4.2$ bin/hdfs namenode -format

15/10/20 14:13:07 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = localhost/10.xx.xx.xx

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 2.6.1

STARTUP_MSG:   classpath = /u01/bigdata/hadoop/etc/hadoop:/u01/bigdata/hadoop/share/hadoop/common/

..................................................................................

OHS 11g Webgate for OAM 11gR2

Install Oracle HTTP Server 11g

Oracle HTTP Server is available as a webserver component in Oracle Web Tier. Download Oracle Web Tier 11g from Oracle.Create a non root user and extract the installer contents from the downloaded Oracle Web Tier zip file and execute runInstaller.

click Next. If you wish to install software updates enter your credentials

select the Install and Configure option and click Next

Be sure you have all the required prerequisites and then click Next.

Create a new Middleware home 

Enter your details to receive security updates.

Select Oracle HTTP Server

Specify Component Details 

Depending on your configuration, select the Auto Port Configuration option or the Specify Ports Using Configuration File 

Verify the installation summary and click Install

Installing Oracle HTTP Server 11g Webgate

Start the Installer by executing  ./runInstaller -jreLoc <WebTier_Home>/jdk

 Click Next to continue.

Specify the Middleware Home and Oracle Home locations.

Click Install to begin the installation.

Click Finish to dismiss the installer.

Post-Installation Steps

Move to the following directory under your Oracle Home for Webgate: <Webgate_Home>/webgate/ohs/tools/deployWebGate and run the following command to copy the required  agent from the Webgate_Home directory to the Webgate Instance location.

For example,

-bash-4.1$ ./deployWebGateInstance.sh -w /u02/app/ssodxbstage/oracle/ohs3/instances/ohs_instance3/config/OHS/ohs3 -oh /u02/app/ssodxbstage/oracle/Oracle_OAMWebGate1

Copying files from WebGate Oracle Home to WebGate Instancedir

Run the following command to ensure that the LD_LIBRARY_PATH variable

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/u02/app/ssodxbstage/oracle/ohs3/lib

cd /u02/app/ssodxbstage/oracle/Oracle_OAMWebGate1/webgate/ohs/tools/setup/InstallTools

On the command line, run the following command to copy the apache_webgate.template from the Webgate_Home directory to the Webgate Instance location (renamed to webgate.conf) and update the httpd.conf file to add one line to include the name of webgate.conf

./EditHttpConf -w <Webgate_Instance_Directory> [-oh <Webgate_Oracle_Home>] [-o <output_file>]

-bash-4.1$ ./EditHttpConf -w /u02/app/ssodxbstage/oracle/ohs3/instances/ohs_instance3/config/OHS/ohs3 -oh /u02/app/ssodxbstage/oracle/Oracle_OAMWebGate1

The web server configuration file was successfully updated

/u02/app/ssodxbstage/oracle/ohs3/instances/ohs_instance3/config/OHS/ohs3/httpd.conf has been backed up as /u02/app/ssodxbstage/oracle/ohs3/instances/ohs_instance3/config/OHS/ohs3/httpd.conf.ORIG

Configure SSO for multiple EBS instances

Often there used to be requirements in enterprises to configure SSO for multiple EBS instances  whether it be Dev, UAT, Prod instances or multiple production environments using the same access manager.In such cases mutiple instances can be secured using one application domain,SSO agent and webgate.

Adding Policies to an existing WebGate and Application Domain

Follow the steps below to add the required policies for additional Oracle E-Business Suite integration to an existing WebGate and Application Domain.

•     Change directories to <RREG_Home>/input.
•     Create a new file named EBS_OAM_PolicyUpdate.xml or use the existing to serve as a parameter file to the oamreg tool. Below is a sample.

 <?xml version="1.0" encoding="UTF-8"?>  
 <PolicyRegRequest>  
   <serverAddress>{protocol}://{oam_admin_server_host}:{oam_admin_server_port}</serverAddress>  
   <hostIdentifier>{Identifier for your existing WebGate}</hostIdentifier>  
   <applicationDomainName>{Identifier for your existing WebGate}</applicationDomainName>  
 </PolicyRegRequest>  

      

• Replace {protocol} with either http, or https if the component has been SSL enabled.
• Replace {oam_admin_server_host} with the fully qualified name for your OAM host.
• Replace {oam_admin_server_port} with the weblogic administration server port (the SSL port if the Admin Server has been SSL enabled).
• Replace {Identifier for your existing WebGate} within both the <hostIdentifier> and <applicationDomainName> elements with the Identifier for your existing WebGate.

Create a new file named ebs.oam.conf to serve as URIs file to the oamreg tool.Change directories to <RREG_Home> and run the following command to add the new policies.

     ./bin/oamreg.sh policyUpdate input/EBS_OAM_PolicyUpdate.xml

When prompted for the admin username and password, enter the credentials for your Oracle Access Manager Administrator, by default user "web logic".

When prompted "Do you want to import an URIs file?(y/n)", enter "y".

Enter the full path for the URIs file that you just created as <RREG_Home>/input/ebs.oam.conf.

The script should complete successfully with a Request summary. Login to the OAM console and check if the URIs are added for the new instance.

Configuring Access gate for multiple EBS Instances

The access gate can be deployed on dedicated managed server as eag_server1 protecting ebs_instance1, eag_server2 protecting ebs_instance2 or can be done on the same weblogic server with different context root.A unique name need to be  used for each application deployment.For example: ebsauth_myEBS1, ebsauth_myEBS2.Also the deployment for each Oracle E-Business Suite environment is performed from a separate file system directory.For example: <MW_HOME>/appsutil/accessgate/ebsauth_myEBS1,<MW_HOME>/appsutil/accessgate/ebsauth_myEBS2.Each Oracle E-Business Suite AccessGate application is tied to a single Apps DataSource configuration during deployment.

 Below entry is required on the OHS with the webgate for redirecting to the corresponding access gate.

   <Location /ebsauth_myEBS1>  
    SetHandler weblogic-handler  
    WebLogicHost eaghost.example.com  
    WebLogicPort 8099  
   </Location>  
   <Location /ebsauth_myEBS2>  
    SetHandler weblogic-handler  
    WebLogicHost eaghost.example.com  
    WebLogicPort 8099  
   </Location>  

Cleanup for Logout from Oracle E-Business Suite

On the WebTier, locate the file oacleanup.html that you copied during Oracle E-Business Suite AccessGate installation to the /public subdirectory on your htdocs root directory.For example $ORACLE_INSTANCE/config/OHS/ohs1/htdocs/public/oacleanup.html

Edit the file and replace CONTEXT_ROOT with the value of the context root for any deployment of Oracle E-Business Suite AccessGate protected by this WebGate. For example:

<script type="text/javascript" src='/ebsauth_myEBS/ssologout_callback?mode=cleanup'></script>

Search for the following line and add a callback for each additional logout callback.

 function doLoad()  
 {  
 logoutHandler.addCallback('/ebsauth_myEBS/ssologout_callback');  
 logoutHandler.addCallback('http://webgatehost2.example.com:7780/ebsauth_myEBS2/ssologout_callback');  

Oracle SOA JVM settings

Upgrading the JVM to the latest version is often required for tuning the performance or upgrading the products like SOA from 11g to 12c which requires higher version of JVM.Even though latest versions have top performance it might not be certified with the product for support. By having proper back of the files we can always fall back to the certified JVM version and reproduce the issue and work with support in case latest versions are not certified. These steps are applicable while starting the servers using the node manager from the admin console.

Upgrading JRockit for SOA

Below are the steps required to upgrade JRockit to a newer version which in our case will be from JRockit R28.2.5 to JRockit R28.3.5.All Java SE Downloads are available on MOS (Doc ID 1439822.1).Since we dont want to replace the java version of the system we have downloaded  and extracted the file to a location with write access(for example /u05/java/jrockit-jdk1.6.0_91).Navigate to the bin folder and run ./java -version to find the version of the newly installed JVM.

• Navigate to $DOMAIN_HOME/bin folder and edit the setDomainEnv.sh file and set the new java home for the variable BEA_JAVA_HOME.
• Another file that needs is commEnv.sh in the $WL_HOME/common/bin location.
• If required change the nodemanager.properties file located at  $WL_HOME/common/nodemanager.

Confirm the changes have taken effect from the node manager and the server logs as below.

Switching the JVM from JRockit to HotSpot

Since JRockit is getting converged with HotSpot as HotRockit and no more update on JRockit after JDK 6 we can also look into options like switching the JVM from one vendor to the other and below are the steps required to switch from JRockit to HotSpot. In that case below are the steps that need to be followed.

Download the latest JRE from Doc ID 1439822.1 and upload the tar to the location where it needs to be installed and run the below command to extract and install.

gzip -dc server-jre-8uversion-solaris-sparcv9.tar.gz | tar xf -


Edit the file commEnv.sh in the $WL_HOME/common/bin location. The varaible JAVA_VENDOR needs to be set to Sun and JAVA_HOME needs to point to the new java home.It is very important to set the Java vendor since there will be many parameters set based on the vendor type.

Navigate to $DOMAIN_HOME/bin folder and edit the setDomainEnv.sh file and set the new java home for the variable SUN_JAVA_HOME.

Also if there are any VM specific parameters that are set as arguments along with memory arguments  that will need to be removed.For example parameter -Xgcprio:pausetime is specific to JRockit.

Make sure from the server logs the JVM has been changed.

Setting Specific memory in SOA domain

Usually there will be many managed servers running in the SOA domain and it will be required to set specific JVM size for each of the servers. This can be achieved by changing the setSOADomainEnv.sh in the $DOMAIN_HOME/bin location as below. 

if [ "${SERVER_NAME}" = "wls_soa1" ] || [ "${SERVER_NAME}" = "wls_soa2" ]; then
  DEFAULT_MEM_ARGS="-Xms1536m -Xmx1536m"
  PORT_MEM_ARGS="-Xms4096m -Xmx4096m"

elif [ "${SERVER_NAME}" = "" ] || [ "${SERVER_NAME}" = "AdminServer" ]; then
  DEFAULT_MEM_ARGS="-Xms1536m -Xmx1536m"
  PORT_MEM_ARGS="-Xms1536m -Xmx1536m"

elif [ "${SERVER_NAME}" = "wls_osb1" ] || [ "${SERVER_NAME}" = "wls_osb2" ]; then
  DEFAULT_MEM_ARGS="-Xms1536m -Xmx1536m"
  PORT_MEM_ARGS="-Xms1536m -Xmx1536m"

elif [ "${SERVER_NAME}" = "wls_wsm1" ] || [ "${SERVER_NAME}" = "wls_wsm2" ]; then
  DEFAULT_MEM_ARGS="-Xms768m -Xmx768m"
  PORT_MEM_ARGS="-Xms768m -Xmx768m"

else
  DEFAULT_MEM_ARGS="-Xms768m -Xmx768m"
  PORT_MEM_ARGS="-Xms768m -Xmx768m"

fi


This can be useful to set specific port for the servers when monitoring the JVM remotely from tools like java mission control.

But setting a larger heap size always is not a good idea for many reasons and a capacity planning needs to be done before changing these values.For better performance there should be a 50 percent free heap space for the process to run.The free heap space can be monitored using tools like Java mission control locally or remotely which should come below 50 percent on full garbage collection. Below is a sample screen shot.

Any customization to the above mentioned files will be overridden if the configuration wizard is run again and will need to be reconfigured. As a option these customizations can be maintained in a different file (say setCustomEnv.sh) and included in setDomainEnv.sh file so that not impacted by any upgrade and can be added back.

Oracle SOA -Payload Size threshold

Oracle JCA Adapters are designed to process large payloads but the BPEL engine consumes huge memory when processing large payloads due to XML conversions which can cause well known OutOfMemory(OOM) and jeopardize the whole system.Setting the payload threshold prevents any data burst from the downstream systems ensuring the JCA Adapters process only payloads that are less than the threshold limit and reject others.

Capping the payload size according to  the capacity of the infrastructure in place is always preferred since the servers will take time to recover from OOM error by garbage collection which is a "Stop the World" activity and no requests will be processed by the server during the time.

Maximum Request size for services

The threshold for maximum request size for any exposed service can be set by using this attribute.The value can be set through the EM console by navigating to the service endpoint -> Service/Reference Properties ->Exposed Service. A default value of -1 represents unlimited which can be set in units of bytes,KB,MB or GB to a suitable value that your infrastructure can handle.

This can be restricted at the DMZ level without the requests reaching the web logic servers if the architecture includes any of the below components.

• Oracle API Gateway
• Web Server that can restrict request size
• Load balancer like F5 BIG IP 

payloadSizeThreshold for adapters

Setting the payload threshold ensures that Oracle JCA Adapters rejects payloads greater than the threshold limit. In case of file and FTP adapters when the native size of the payload is not available and if the specific adapter does not use the native translation library, you cannot enforce the payload size threshold limit. For example, in case of xml-debatching, where the Oracle File and FTP Adapters pass a chunk of file content and the actual native size is not known, payload size threshold limit cannot be used.

To set the global property for capping payload size login to the EM console -> soa-infra   -> Administration -> System MBean Browser - > adapter and set the value for DefaultPayloadSizeThreshold . As per the documentation DefaultPayloadSizeThreshold  works only for the inbound adapters.There is no attribute as payloadSizeThreshold which seems to be a bug and DefaultPayloadSizeThreshold works for both inbound and outbound operations.

This threshold can be increased/decreased at adapter service level by overriding the values in composite.xml which takes precedence over the global settings.

 <reference name="insert" ui:wsdlLocation="insert.wsdl">   
 <interface.wsdl interface="http://xmlns.oracle.com/pcbpel/adapter/db/Application3/One2ManyJoining/insert#wsdl.interface(insert_ptt)"/>   
 <binding.jca config="insert_db.jca"/>   
 <property name="payloadSizeThreshold" type="xs:string" many="false" override="may">100</property> </reference>   

Below are some of the options when it is required to process huge data using database  adapter.

• MaxRaiseSize and MaxTransactionSize can be used while using database polling to restrict the number of records returned from the adapter.
• Setting the audit level to minimal can enhance  the processing of large data with a drawback of payload not getting saved to the database
• When processing large payload using XSLT set "streamResultToTempFile" to yes to avoid OutOfMemory error.But assign activities always perform better.
• Parking Lot pattern can be used to throttle the messages processed.See Throttling in SOA Suite via Parking Lot Pattern at http://www.ateam-oracle.com/throttling-in-soa-suite-via-parking-lot-pattern/  for how to implement in SOA.

Other adapters like AQ,MQ,File,FTP etc... have their own parameters that can be tweaked to control the payload size when used for inbound operations.

Zero Sign-On (ZSO) or IWA for IIS 8 applications using OAM11GR2

Both  Zero Sign-On (ZSO) and Single Sign-On (SSO) means that user has one username and password (e.g.Active Directory username and password) for the SSO enabled application but ZSO  authenticates seamlessly without prompting for a username and password using  the desktop credential which is achieved using kerberos protocol.Below are the steps to configure ZSO for .NET websites running on IIS server using Oracle access manager.

Environment

OAM :11.1.2.2.0 

Web server:IIS 8 on Windows Server 2012 R2 

Webgate: 11.1.2.2.0

Prerequisite

• Install Visual C++ Redistribution for Visual Studio 2012 Update 4 or else the files will not be copied properly during installation
• Install a 64-bit Java runtime environment (JRE), 1.6 or higher  
• It is recommended to run the command prompt as administrator and execute all the scripts
• Make sure to provide full access for the middleware home and the webgate instance folders.
• Make sure the site is deployed on IIS server and able to list  using the command. 

      %systemroot%\System32\inetsrv>appcmd.exe list sites 

Installing IIS 11g WebGate

Extract the contents of the webgate.zip file to a directory, Go to the Disk1 and run the below command.

setup.exe -jreLoc 64_bit_jre_location

Click Next to continue.

Click Next to continue.

Specify the Middleware Home and Oracle Home locations.

Click Install to begin the installation.

Click Finish to dismiss the Installer.

To deploy the WebGate instance , Go to the webGate_Oracle_Home\webgate\iis\tools\deployWebGate  directory and run the following command

deployWebGateInstance.bat -w WebGate_Instancedir -oh WebGate_Oracle_Home -ws WebServer

To run the ConfigureIISWebGate.bat tool,go to the WebGate_Home\webgate\iis\tools\ConfigureIISConf and run the below command. 


ConfigureIISWebGate.bat -oh c:\WGHome -w c:\WGInstance -site "mysite"

Make sure the webgate.ini has some entry as below for the registered web gate instance.19 represents the site id protected by the access gate.

Also make sure the ISAPI filters are added pointing to the webgate.dll as below.

Register the WebGate using RREG

The web gate registration can be done from the OAM console or the rreg scripts. For registering using the scripts navigate to OAM_REG_HOME/bin and execute the below command.

$ ./oamreg.sh inband input/test_OAMRequest.xml 

Copy the files generated in the RREG_Home\output\Agent_ID  to the WebGate_Instance_Home\webgate\config directory

Make sure the sso agent is registered as 11g webgate.

Configure OAM to use WNA

• Create a  user in Microsoft Active Directory for example oamuser.

• Run ktpass on the KDC server to create the SPN (service principal name)  and associate it with this user. For example

ktpass -princ HTTP/myhost.mydomain.com@DOMAIN.COM -pass ***** 

-mapuser oamuser -out D:\etc\oam.keytab

where myhost.mydomain.com is the FQDN of the host where access manager is running or the host name of the loadbalancer VIP in case of OAM cluster.

• Edit the /etc/krb5.conf file to include the domain and the KDC server.

• Configure the Kerberos authentication scheme to use WNA by Logging in to the OAM console ->Launch Pad ->Authentication schemes >KerberosScheme and change the challenge method to WNA

Login to the OAM console ->Launch Pad ->Authentication Modules >Kerberos and change the default values to the actual values.

• Configure the application domain protecting the resource to use the Kerberos authentication scheme.

• Register the active directory as the identity store and make this as the primary user identity store for Oracle Access Manager.

After you start the IIS Web Server (iisreset), log in to the site by using the following URL without entering any credentials.

http://myhost.domain.com:port