Discussion

Parallel Processing using Agents

Hi Gurus,
I am working in Healthcare industry. Our team processes T837 X12 format files.
we are trying to develop an application using PEGA to process the T837 files.
We are new to PEGA. Our team is trying to develop the application to parse the
T837 files using PEGA supplied Healthcare frame work. We are trying to develop
our application to scale it. We have made a minor modification to one of the
existing activity to achieve parallel processing. T837 Files comes with multiple
Transaction groups and each Transaction group contains multiple claims. We are
creating the work objects at Transaction group level and processing them using an
agent which runs at 30 sec interval. With one agent it took 30 minutes to process
a sample file. Now we have created two agents to increase the processing speed
and distributed the work objects equally to two agents using precondition. Our guess
was that the processing time will be reduced by half. But the processing time was not
reduced much and it took 25 minutes to process the same sample file. Again we have
increase the agents to 5 and used the same sample file. But still the processing time
was 25 minutes. We have not understood why the processing time is not getting reduced
even though we are distributing the work objects equally to all agents. Any help would
greatly appreciated regarding this issue. If any body likes to volunteer to help us to
solve this issue and requires more details, please contact me at dsingam@amerigroupcorp.com

Thanks in advance.

Jay

**Moderation Team has archived post**

This post has been archived for educational purposes. Contents and links will no longer be updated. If you have the same/similar question, please write a new post.

Comments

Keep up to date on this post and subscribe to comments

September 12, 2012 - 9:40pm

If you are using two agent's and both are picking the case at same time or fraction of seconds difference it should process two objects in x minutes in parallel. You may want to log the time when agent is picking up the case and which agent is picking the case to make sure your precondition work and the same agent is not processing all the cases.

But I would like to know how many nodes do you have in your environment .
Which agent mode are you using ?
If you have multiple nodes then you can use standard agent which will process the work cases in parallel while running on different nodes .

September 13, 2012 - 9:43am

Anand,
Thank you for the reply. We have 2 nodes in the Dev environment.
We are running the agents in standard mode. Increasing the nodes is our last option. We are trying to prove that the processing time can be reduced by increasing the agents. We are allocating the objects to different agents based on precondition and confirmed that the agents are processing different work objects and not the same WO. If you need more details I can send the screen shots.

quote:


Originally posted by AANAND84

If you are using two agent's and both are picking the case at same time or fraction of seconds difference it should process two objects in x minutes in parallel. You may want to log the time when agent is picking up the case and which agent is picking the case to make sure your precondition work and the same agent is not processing all the cases.

But I would like to know how many nodes do you have in your environment .
Which agent mode are you using ?
If you have multiple nodes then you can use standard agent which will process the work cases in parallel while running on different nodes .


September 14, 2012 - 12:27am

Did you confirm the time when both agents pick up the case. Look at the time , try to queue work cases at same time and check when both agents pick up the case and you log few messages in your activity to to confirm the time and step that gets executed.

September 18, 2012 - 8:53am

quote:


Originally posted by AANAND84

Did you confirm the time when both agents pick up the case. Look at the time , try to queue work cases at same time and check when both agents pick up the case and you log few messages in your activity to to confirm the time and step that gets executed.

I am facing the same issue. My agents are in advanced node. I have 4 agents in 2 node. We are assigning the rows with a column which has designated number of rows that will be only picked with particular node/agent. Say I have 8000 records to create work object I mark them as N1A1,N1A2,N1A3,N1A4,N2A1,N2A2,N2A3,N2A4. 1000 each

This is not working as per expectation and it is processing as fast as one agent only. Please suggest.

October 18, 2012 - 1:18pm

Hi,

Finally, We have solved our issues about using agents. At present our configuration is 4 nodes each with 15 agents and we are able process 160K claims in couple of hours. We have used 15 agents like agent1, agent2, agent3... and so on, calling the same activity which processes the objects in the queues. Each agent assigned the object using our business logic in an activity. Intially we had issues with database connection. Through the connection was working but the parameters in the connection string were not set properly which made the process to run for hours. Once we figured out the correct connection string its reduced to couple of hours. While working on this issue we learned lot of lessons about using the agents. ;-)

 

October 18, 2012 - 1:22pm

Hi Solu_Abhi,

The provided details are not enough to understand the issue. Send me more details about the agent and node configuration and the activity that is calling the agents clearly, possibly with screen shots.

jay

dsingam@amerigroucorp.com

 

October 18, 2012 - 1:23pm

repost - ingore

October 16, 2015 - 10:05am

Hi JaySetty,

 

We are working on simillar issue 837 edi files, but our issue is generating the 837 output file. How did you create the output file ?

I used the out of the box activity GenerateOutputFile but it's not consistent, the file is sometimes not generated and no errors on tracer or in the logs. So was curious to know how did your team implement that process ?

 

Thanks