Question

Data flow not created

Data flows (Batch Process) are created in our dev environment either from agent activity or direct activity execution. Whereas in our QA environment data flows are not created through both the approaches (Agent/activity).

There are no differences in system configuration and DSS in both the environment. We are not able to trace the Dataset.

DataFlow rule on the explicit run, the Batch process is created with auto-generated work object id.

From Activity DataFlow-Execute method is executed successfully but the batch process is not created in the decisioning landing page. (which is working in Dev environment and able to see the batch process DF id in decisoning landing page).

***Edited by Moderator: Pallavi to update platform capability tags***

***Edited by Moderator Marissa to update SR Details***

Group Tags

Comments

Keep up to date on this post and subscribe to comments

May 14, 2019 - 7:25am

Have you got a running DF process in QA? What version of Pega are you on. For example, here is a screenshot from 7.3.1. What does it show to the right if you click on status "Normal"?

 

May 14, 2019 - 9:38am
Response to PaulGentile_GCS

Hello Paul, 

Thanks for your response. We are using Pega 7.3.1 version. 
In decisioning services landing page, all the node status are normal. 
 

 

May 14, 2019 - 7:28am

Are you able to run the DF manually in QA? If not, any errors?

May 14, 2019 - 9:40am
Response to KevinZheng_GCS

Hello Kevin, 

From the Decisioning landing page, on click of "New" DF (Batch process) is created successfully.

May 16, 2019 - 5:20am

After enabling logger: com.pega.dsm.dnode.impl.dataFlow

I have seen the below errors:

2019-05-16 05:17:04,511 [tp-nio-5451-exec-193] [  STANDARD] [                    ] [           XXX:01.01] (      internal.mgmt.Executable) ERROR xxxx-xxx|10.119.20.xxx - Only authenticated client may start this activity: RULE-OBJ-ACTIVITY PEGA-DM-DDF-WORK ADDWORK #20170501T125928.649 GMT ReqID=BHT6I54U4B5H9PN56B0M3LLRNS4KYG36I 
com.pega.pegarules.pub.PRRuntimeException: Error: You lack access required to execute RULE-OBJ-ACTIVITY PEGA-DM-DDF-WORK ADDWORK #20170501T125928.649 GMT.  
	at com.pega.pegarules.session.internal.mgmt.Executable.updateStepStatusAndLog(Executable.java:912) ~[prprivate.jar:?] 
	at com.pega.pegarules.session.internal.mgmt.Executable.activityProlog(Executable.java:844) ~[prprivate.jar:?] 
	at com.pegarules.generated.activity.ra_action_addwork_db4db4af2ddc4865d9e3d30003879de0.perform(ra_action_addwork_db4db4af2ddc4865d9e3d30003879de0.java:53) ~[?:?] 
	at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3597) ~[prprivate.jar:?] 
	at com.pega.dsm.dnode.impl.dataflow.activity.PRPCActivities$PRPCActivity.execute(PRPCActivities.java:100) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.impl.dataflow.activity.PRPCActivities$PRPCActivity.access$400(PRPCActivities.java:69) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.impl.dataflow.activity.PRPCActivities.addWork(PRPCActivities.java:20) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.impl.dataflow.service.persistence.db.RunDbPersistence$WorkObjectCreator.create(RunDbPersistence.java:400) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.impl.dataflow.service.persistence.db.RunDbPersistence.getOrCreate(RunDbPersistence.java:82) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.api.dataflow.service.DataFlowRunFactory.from(DataFlowRunFactory.java:51) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.impl.dataflow.service.DataFlowServiceImpl$6.run(DataFlowServiceImpl.java:272) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.impl.dataflow.service.DataFlowServiceImpl$6.run(DataFlowServiceImpl.java:269) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:52) ~[dnode-7.3.1.jar:?] 
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:202) ~[dnode-7.3.1.jar:?] 
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1327) ~[prprivate.jar:?] 

June 17, 2019 - 3:28pm

We've seen a very similar issue like this with a client who wasn't using PaidMedia but had found this exception was related to a data flow for that functionality.

In that scenario, the logs provided more details directly before the above exception was seen. 

Problem in executing BatchData-Decision-DDF-Run-Partition: code: 1400 SQLState: 23000 Message: ORA-01400: cannot insert NULL into ("PRPC_DATA"."PR_DATA_DECISION_DF_PART"."PXWORKID")

Please check for an occurrence like this. The root cause of prior issue was "Update Match Rate activity runs in agent context every one hour, the Update Match Rate activity triggers the data flow run by using the pxRunDDFWithProgressPage API to create audience snapshot. User might have not enabled the Pega Paid Media capabilities; hence the exceptions are being noticed."

The solution provided was:

"Perform the following local-change: "UpdateMatchRate" activity is required if User have enabled paid media capabilities. User can safely disable "UpdatePaidMediaMatchRate" agent since the paid media capabilities are not enabled".

 

June 17, 2019 - 3:52pm
Response to PaulGentile_GCS

Thanks of the response @PaulGentile_GCS

I have created SR (SR-D16324) regarding the above issue, Pega support team is looking to find the RCA on it.

 

Regards,
Naveen

August 13, 2019 - 10:50am

Hello,
we had a similar behavior here. Is there an answer to the SR already?

 

August 13, 2019 - 10:56am
Response to RonnyT46

Hi @RonnyT46,

This SR is not resolved yet. This is with our Engineers.

Marissa | Community Moderator | Pegasystems Inc.