Question

Thread dump during performance testing

Hi Team,

During performance testing, I am below thread dumps in Pega logs. Can you please help in identifying the exact reason about why it is happening? Please find below short snippet on thread dump as it is of 5000 line.For more info please find attached file

Thanks for help

2019-10-16 20:53:58,972 [ WebContainer : 17] [ ] [ ] [ ] (.timers.EnvironmentDiagnostics) INFO pega.saifg.rbc.com|10.60.36.5 - --- Thread Dump Starts ---

Full Java thread dump with locks info

"P=817818:O=0:CT" Id=1 in TIMED_WAITING (running in native)

BlockedCount : 10, BlockedTime : -1, WaitedCount : 319, WaitedTime : -1

at java.lang.Thread.sleep(Native Method)

at java.lang.Thread.sleep(Thread.java:941)

at com.ibm.ws.runtime.WsServerImpl.main(WsServerImpl.java:715)

at com.ibm.ws.runtime.WsServer.main(WsServer.java:59)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

at java.lang.reflect.Method.invoke(Method.java:508)

at com.ibm.wsspi.bootstrap.WSLauncher.launchMain(WSLauncher.java:234)

at com.ibm.wsspi.bootstrap.WSLauncher.main(WSLauncher.java:96)

at com.ibm.wsspi.bootstrap.WSLauncher.run(WSLauncher.java:77)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

at java.lang.reflect.Method.invoke(Method.java:508)

at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)

at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)

at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)

at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)

at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)

at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

at java.lang.reflect.Method.invoke(Method.java:508)

at org.eclipse.core.launcher.Main.invokeFramework(Main.java:340)

at org.eclipse.core.launcher.Main.basicRun(Main.java:282)

at org.eclipse.core.launcher.Main.run(Main.java:981)

at com.ibm.wsspi.bootstrap.WSPreLauncher.launchEclipse(WSPreLauncher.java:406)

at com.ibm.wsspi.bootstrap.WSPreLauncher.main(WSPreLauncher.java:169)

Locked synchronizers: count = 0

"JIT Compilation Thread-0" Id=4 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT Compilation Thread-1 Suspended" Id=5 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT Compilation Thread-2 Suspended" Id=6 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT Compilation Thread-3 Suspended" Id=7 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT Compilation Thread-4 Suspended" Id=8 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT Compilation Thread-5 Suspended" Id=9 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT Compilation Thread-6 Suspended" Id=10 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT Diagnostic Compilation Thread-7 Suspended" Id=11 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"JIT-SamplerThread" Id=12 in TIMED_WAITING

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"IProfiler" Id=13 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"Signal Dispatcher" Id=2 in RUNNABLE (running in native)

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

at com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)

at com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)

Locked synchronizers: count = 0

"Concurrent Mark Helper" Id=14 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"GC Slave" Id=15 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

"GC Slave" Id=16 in RUNNABLE

BlockedCount : 0, BlockedTime : -1, WaitedCount : 0, WaitedTime : -1

Locked synchronizers: count = 0

***Moderator Edit-Vidyaranjan: Updated Platform Capability***

Comments

Keep up to date on this post and subscribe to comments

October 18, 2019 - 2:52am

Hello, 

I don't know if this extract would be sufficient. We might need the full log and also some details about your performance testing how it has been done. 

I would guess you have a Commit failing due to locking issues outside Pega but I'm not sure:

at com.ibm.db2.jcc.am.ki.executeBatch(ki.java:3045)
      - locked com.ibm.db2.jcc.t4.T4XAConnection@e8fe5b55
    at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecuteBatch(WSJdbcPreparedStatement.java:1145)
    at com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.executeBatch(WSJdbcStatement.java:855)

 

October 18, 2019 - 5:49am

Hi Marc,

We are seeing commit error as well. Please find attached complete log file for you reference.

Thanks,

Hemant

October 18, 2019 - 6:12am
Response to HemantK9

Have you tried to update statistics at the database level?  

October 18, 2019 - 7:11am

I am not sure which database statistics you are pointing. Can you please be more specific?