Friday 23 August 2013

Migrating an OBIEE Server to a Different IP Address


I had an OBIEE server moved to a different IP address recently and after the move the following errors could be observed in the logs:

<21-Aug-2013 12:40:22 o'clock BST> <Warning> <oracle.adfinternal.view.faces.partition.FeatureUtils> <ADF_FACES-30130> <Ignoring feature-dependency on feature "AdfDvtCommon".  No such feature exists.>
<21-Aug-2013 12:40:22 o'clock BST> <Warning> <org.apache.myfaces.trinidadinternal.config.GlobalConfiguratorImpl> <BEA-000000> <Configurator services already initialized.>
<21-Aug-2013 12:40:27 o'clock BST> <Notice> <Log Management> <BEA-170027> <The Server has established connection with the Domain level Diagnostic Service successfully.>
<21-Aug-2013 12:40:28 o'clock BST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN>
<21-Aug-2013 12:40:28 o'clock BST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RESUMING>
<21-Aug-2013 12:40:28 o'clock BST> <Emergency> <Security> <BEA-090087> <Server failed to bind to the configured Admin port. The port may already be used by another process.>
<21-Aug-2013 12:40:28 o'clock BST> <Error> <Server> <BEA-002606> <Unable to create a server socket for listening on channel "Default". The address <OLD_IP> might be incorrect or another process is using port 7001: java.net.BindException: Cannot assign requested address: JVM_Bind.>
<21-Aug-2013 12:40:28 o'clock BST> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason: Server failed to bind to any usable port. See preceeding log message for details.>
<21-Aug-2013 12:40:28 o'clock BST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<21-Aug-2013 12:40:28 o'clock BST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<21-Aug-2013 12:40:28 o'clock BST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
[Wed Aug 21 12:40:32 2013] [I] [ExitHandler] Fire (-1)
[Wed Aug 21 12:40:32 2013] [I] [ServiceStart] done waiting for multiple events. Wait=0
[Wed Aug 21 12:40:32 2013] [I] [ServiceStart] Informing SCM about SERVICE_STOP

Thankfully I have seen this error before. I performed the following steps to remediate:

Changed IP address to FQDN in the following files:

E:\Oracle\Middleware\instances\instance1\config\OPMN\opmn\instance.properties
E:\Oracle\Middleware\instances\instance1\config\OracleBIApplication\coreapplication\security\security.xml
E:\Oracle\Middleware\instances\instance1\config\OracleBIPresentationServicesComponent\coreapplication_obips1\instanceconfig.xml
E:\Oracle\Middleware\instances\instance1\config\OracleBISchedulerComponent\coreapplication_obisch1\instanceconfig.xml
E:\Oracle\Middleware\instances\instance1\config\OracleBIServerComponent\coreapplication_obis1\NQSConfig.INI
E:\Oracle\Middleware\wlserver_10.3\common\nodemanager\nodemanager.properties

Then altered the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\beasvc bifoundation_domain_bi_server1\Parameters
Altered CmdLine parameter to change the value of -Dweblogic.management.server to have the FQDN instead of the IP address.
Also altered CmdLine parameter to add -Dweblogic.ListenAddress="<FQDN>" just before "weblogic.Server".

Then altered the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\beasvc bifoundation_domain_AdminServer\Parameters
Altered CmdLine parameter to add -Dweblogic.ListenAddress="<FQDN>" just before weblogic.Server.

Node Manager was restarted, then the AdminServer and bi_server1 started up fine.

Please note the steps involving the registry only work for environments which have the OBIEE services created in the services.msc panel.

For environments without the services I think you would have to change the "Listen Address" for the AdminServer and bi_servers in the WebLogic Admin Console to the FQDN before migrating the VMs between environments. If you didn't have a chance to do this you would have to edit the E:\Oracle\Middleware\user_projects\domains\bifoundation_domain\config\config.xml file.

Hope that helps some people!

2 comments:

  1. It depends upon the scale of the changes in the upgrade or the amount of regression testing you want to do, this may be a factor.In 11.1.1.3 -> 11.1.1.5 -> 11.1.1.6, the in-place upgrade was complex and infamous for being buggy.

    Thanks
    Silvester Norman

    Change MAC Address

    ReplyDelete
  2. I haven't seen this issue in upgrades, I have seen it in migrations between DNS domains and when machine IP addresses change.

    ReplyDelete