Discussion

Introducing Delay in File Listener

I know there has been a few discussions in the PDN about how a file listener (only one running on one node) would process the files if there are multiple files placed in the folder. My questions is whether it is possible to introduce a delay between the processing of multiple files in a file listener.

Scenario:5 files are put in the folder which is monitored by the file listener. Each file has a specific number of records. I would like to have the processing sequence as follows:

  • FL1 picked up and the service file processes the records.
  • there is a gap of a minute or so then FL2 is picked up for processing and so on

Is this practically possible in Pega? I have tried using the Idle Processing section but to no avail.

the only option I see is to have a job running on the OS where Pega is installed, and this job would send the file one at a time to the folder which Pega is monitoring.

any light in this direction would be very appreciated.

**Moderation Team has archived post**

This post has been archived for educational purposes. Contents and links will no longer be updated. If you have the same/similar question, please write a new post.

Comments

Keep up to date on this post and subscribe to comments

May 29, 2015 - 12:51pm

Hi Anoojit,

Looking at the FileListener code now and unfortunately i don't see any logic to put a delay in between processing of each file.  Basically we just loop through each matching file in the directory and process the file one at a time without any delay.  As you mentioned above, controlling it from the OS level may be the way to go for now.

=waikei

May 29, 2015 - 2:21pm

I don't understand the requirement very well. Do these files need to be processed in a certain order and does file listener need to apply different logic on different files?

May 29, 2015 - 2:48pm

Hi Anoojit,

Wai-kei is right there is nothing in the file listener code to configure a delay. The file listener is greedy and will continue processing until all matching files are gone.

Just for completeness can you tell us what your PRPC version is?

I also have a question regarding the use case. It is not every day we get a request to slow things down. Can you talk more about that as well?

May 30, 2015 - 1:27am
Response to JimKelly_GCS

Hi,

The requirement is that the folder which Pega listens to would be filled by an external application. That application would pump in files at regular intervals and it may turn out that at a particular point in time there are more than 1 file in the folder. Our application interacts with a plethora of external systems which are load sensitive, so if we pump in large number of requests from multiple files into them , they would potentially become unresponsive. I understand that this limitation / restriction is on the external system, and not Pega, but I was looking for a mechanism by which I can say that I will process each file after an interval od 1 minute between them so that the load is regulated.

We are using Pega 7.1.7

However I did see that if I put in a Wait method in the Idle Processing, there is indeed a small delay (whatever I put as waitseconds) between processing of the file contents. But I am not sure if it is a good practice to have waiting threads in the system, specially if tomorrow we have 5 more listeners with similar requirements.

May 30, 2015 - 8:30am
Response to DHARA001

Here at Pega, we depend on an internal  app (are we allowed to say its name here ?) that keeps track of all customer cases.  That app depends on many of Pega’s core features, so when I’m wondering how to do something specific, I often try to think examples where that app already does it.  (I also often think about the dark side which is, when someone mentions a suspected bug and I’m wondering how to reproduce it, I think about situations where I may have encountered that bug in our internal app).

That internal app generates a note on a case that contains email that is received for the case (identified by the case number appearing in the subject line of the email).

If the case is locked, and the app can not attach the note, it waits several minutes before trying again.    Let’s think about how it orchestrates this waiting several minutes and trying again (or merely ask the developers how they do it!), since it seems to me that that waiting is exactly what you’re asking for in this discussion.  In your situation, instead of a case being locked being your signal to wait, it sounds like you merely want to look at the system clock, and if not enough time has passed yet, you want to wait.

/Eric

June 1, 2015 - 12:11pm

You should be able to throttle the speed at which File Listener processes the files using

Polling Interval (seconds)

Enter the number of seconds this listener waits before it checks for newly arrived files.

and Idle Processing

Activity Class

SmartPrompt

Optional. To identify an activity that runs after the processing of a file is complete, enter the Applies To key part here.

Increase the polling interval and specify an activity that runs after every file is processed.

June 3, 2015 - 5:40am
Response to VipinKumar38163630

Hi Vipin,

Thanks for the post. Actually I was kind of able to control the pace at which the file listener picks up and process the file with the idle processing. I introduced a wait on the thread for a definite period of time.