Discussion

Data store and stream nodes not joining on Oracle WebLogic 12.2.1.3 after Patch Set Update

With recent Oracle JDK versions (JDK 7 Update 131 or later/JDK 8 Update 121 or later) and WebLogic Server October 2017 PSU or later, WebLogic implements JEP 290 support which increases security on RMI (see Oracle Fusion Middleware - Configuring a Custom JEP 290 Deserialization Filter).

After starting the Cassandra JVM, the Pega Platform tries to connect on local Cassandra JMX port (default 7199) to control the instance.

With WebLogic default security configuration, despite Cassandra JVM being correctly started and the JMX port being open and reachable (note it is local and bound to localhost), the Platform will indefinitely wait for connection to the JMX port to successfully complete.

In the WebLogic managed server log, the following message will show every second:

####<Mar 14, 2019 5:00:08,827 PM CET> <Info> <java.io.serialization> <***> <***> <DSM init service: DDS.Default> <<anonymous>> <> <9115d352-aa60-436c-aedf-66c65cc4ea3f-0000000d> <1552575608827> <[severity-value: 64] [rid: 0:3:1:1:8] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000000> <ObjectInputFilter REJECTED: class java.rmi.server.RemoteObject, array length: -1, nRefs: 6, depth: 3, bytes: 169, ex: n/a>

To allow the Platform to successfully control the Cassandra instance on its local JMX port, add the following to the Pega Platform JVM parameters:

-Dweblogic.oif.serialFilterMode=combine
-Dweblogic.oif.serialFilter=java.rmi.server.RemoteObject

Note that the class java.rmi.server.RemoteObject is the one shown in the log message.

Similarly, after starting the Kafka JVM, the Pega Platform tries to connect on local Kafka JMX port (default 9999) to control the instance. In this case, add the following to the Pega Platform JVM parameters:

-Dweblogic.oif.serialFilterMode=combine
-Dweblogic.oif.serialFilter=java.rmi.server.RemoteObjectInvocationHandler;java.rmi.server.RemoteObject

-

Your friendly neighborhood Technical Architect

Comments

Keep up to date on this post and subscribe to comments

August 30, 2019 - 3:37am

Thanks very much for sharing this. It has been very helpful!