Discussion

Split processing to run in parallel

We have an agent that gets data from an upstream application.In the agent activity, we are calling 2 activities - PAT,PNT. Since PAT takes a long time to execute, PNT has to wait until that time and our processing time becomes huge, around 7hrs. Our requirement is to put these 2 processes in parallel so that while PAT is executing, PNT also runs and completes its own processing. We thought of making our process a flow based one by using Split/Join; however since agent activity is the first activity that gets called, pega work objects(WOs) are not formed at that stage and we are unable to work with flows. Is there another way to process these 2 activities in parallel. FYI, PAT does a RDB-List to extract WOs with TS prop = 'AT' and PNT does a RDB-List to extract WOs with TS prop = 'NT' and then these results are processed and the main WO flow starts. Thanks.

***Updated by Moderator: Marissa to close post***
This post has been archived for educational purposes. Contents and links will no longer be updated. If you have the same/similar question, please write a new post.

Comments

Keep up to date on this post and subscribe to comments

November 24, 2008 - 12:23am

quote:


Originally posted by KSHIRSAS

We have an agent that gets data from an upstream application.In the agent activity, we are calling 2 activities - PAT,PNT. Since PAT takes a long time to execute, PNT has to wait until that time and our processing time becomes huge, around 7hrs. Our requirement is to put these 2 processes in parallel so that while PAT is executing, PNT also runs and completes its own processing. We thought of making our process a flow based one by using Split/Join; however since agent activity is the first activity that gets called, pega work objects(WOs) are not formed at that stage and we are unable to work with flows. Is there another way to process these 2 activities in parallel. FYI, PAT does a RDB-List to extract WOs with TS prop = 'AT' and PNT does a RDB-List to extract WOs with TS prop = 'NT' and then these results are processed and the main WO flow starts. Thanks.

As both activities are doing RDB-List. Will it be a better option if you check the "Run-In-Parallel" option on the RDB List. Correct me

November 24, 2008 - 4:29am

quote:


As both activities are doing RDB-List. Will it be a better option if you check the "Run-In-Parallel" option on the RDB List. Correct me

The problem is the access time taken to proces the records and not just filtering them using RDB-List. What you are suggesting will only fetch us the second set of records for processing; however we will be able to proces them only sequential; which is already happening now and consuming time.

February 13, 2009 - 11:40am

can threading be used here to run in parallel and process multiple records in parallel ?????

queueBatchActivity() ????

February 19, 2009 - 6:32pm

thought this may be helpful. have a look.

http://pdn.pega.com/DevNet/PRPCv4/KB/13872.asp
/prhelp/definitions/t/threadpage.htm

November 19, 2008 - 8:22am

You can use BRANCH instead of CALL instruction to call the activities in parallel. The limitation being that the control does not return to the calling activity.

quote:


Use the Branch instruction to cause the current activity to find another specified activity and branch to it without a return. When the system executes a Branch step, control transfers to another activity found through rule resolution.

You can pass parameters to the target activity, or you can share the current activity parameter page with the target activity.

example. Branch Data-Admin-Operator-ID.Analyze


Check PRPC Help for more information.

November 19, 2008 - 8:48am

AGGARWAA1 wrote: You can use BRANCH instead of CALL instruction to call the activities in parallel.

Where did you read that?
That is not the case. Branch does NOT spin off a parallel thread.
Nor does Split/Join.
If there is documentation which suggests that, please let us know.

KSHIRSAS: why do you see that parallelism would help? Are these processes waiting on an external resrouce? You could also run PNT first, too...

Jon Garfunkel
Pegasystems

November 19, 2008 - 10:56pm

Thanks for the responses...
I totally agree with Jon. Branch would not start a new parallel thread.

KSHIRSAS: why do you see that parallelism would help? Are these processes waiting on an external resrouce? You could also run PNT first, too...
>>> Both PNT() and PAT() process more than 15K records daily taking approx 5 and 2.5 hrs respectively; and help in creating Pega WOs. It is not like which one runs first or second; the only thing is that if we run them in parallel, the time would reduce to 5 hrs which will tremendously help us improve our performance.

November 19, 2008 - 11:45pm

quote:


Originally posted by GARFJ

AGGARWAA1 wrote: You can use BRANCH instead of CALL instruction to call the activities in parallel.

Where did you read that?
That is not the case. Branch does NOT spin off a parallel thread.
Nor does Split/Join.
If there is documentation which suggests that, please let us know.


Sorry, I think I misunderstood the documentation for BRANCH. I thought that BRANCH calls the activity and continues with the current one.

Pega
November 19, 2008 - 11:46pm

It sounds like there are two phases here: an agent that wakes up and receives data from another application (how does it receive it? could this be done with a listener?), and then there are two independent processing activities that use the data. Is that correct?

If they are truly independent of each other, and there is no possibly of duplicate work, contention between the two, or collisions as they are processing, why not have 1 agent that receives that data, creates work items, and then have 1 agent for PAT and another for PNT?

This assumes that the agent that initially receives the data is capable of importing into PRPC in such a way (creating work items using standard BPM interfaces?) that PAT and PNT can wake up regularly as separate agents and query for any new work to process without requiring that the 1st agent (the importer) be fully complete.

-Tim(pega)

November 20, 2008 - 6:07am

There is 1 agent(A1) picking up the data and other setting different props for processing the records and creating Pega WOs. Second agent(A2) calls PAT followed by PNT. These 2 activities then do a RDB-List, get their respective data sets and start the main flow with a WO ID like S-1234856. First agent is getting the data from the upstream application using Unix scripts. Is it possible that we add 1 more agent(A3) to the queue and modify the scipt to start both agents(A2&A3) in parallel when the previous agent(A1) has brought the data in to our system? Then we will allow A2 to call PAT & A3 to call PNT, thus to run both PAT and PNT in parallel.

November 20, 2008 - 6:38am

I think you can achieve the requirement through the flow base also.You said that in the agent2 you are executing the PAT followed by PNT.In the second agent you can call an activity which has a method to call the flow(Call ADD method),The Called flow must have a Split/Join shape through which you can run the 2 activities parallely.

Hope i understood your problem......

Correct me if am wrong..

Cheers....[:)]

November 20, 2008 - 7:03am

vspalukuru,
The problem why we can't use flows is that the Pega WOs do not get created at that point. So looking for an alternative solution. In fact the 2 activities PAT, PNT only are responsible for creation of WOs.

November 20, 2008 - 7:13am

Is it possible to do like this --

get DATA (A1) >> onethread >> A2 > calls PAT > processes records to create Pega WOs with TStatus = AT

get DATA (A1) >> secondthread >> A3 > calls PNT > processes records to create Pega WOs with TStatus = NT

...with A2 and A3 operating in parallel. All the records will go to both agents and depending up on PAT or PNT, filtering will be done.