Offline Data Pages do not accept Params: need advice.

The customer has a requirement to set up offline data on mobile devices (both iOS and Android).
The backend data set is very large (380k [more? records]; probably too large for performance reasons on the back-end PRPC system and probably too large for the Mobile Device storage.

Previously: Parameters were specified on the Data Page in an attempt to cut down the recordset to a few hundred records.
This approach failed (the sync process would 'hang'): and Pega SMEs were engaged.

SMEs identified the reason: Parameterized Data Pages cannot be used in this context (the params are ignored, or not available in this context).
Therefore the entire data-set was downloaded to the mobile device - unfiltered. (At least on the 'initial sync').


**Moderation Team has archived post**

This post has been archived for educational purposes. Contents and links will no longer be updated. If you have the same/similar question, please write a new post.

Group Tags


Keep up to date on this post and subscribe to comments

August 25, 2016 - 1:11pm

The solution depends on getting the data that is actually needed to be worked on.

For example in this case instead of getting all the flight data for all the passengers for 2 years down to the client for a crew member to access in offline, we need to refine the filter criteria at point of retrieval on server during the packaging sync request::

Suggestions - 

1. Add a new column for the crew member to the flight master data table (if possible) so the sync gets filtered data only for the flight manifests of which the crew member is part of. It may not be possible to change existing tables / schema. Instead we can do a Join on the crew member roster with the flight passenger data .


2. Add the  condition to the report definition to essentially get only data for  the current day (or maybe day before and day ahead) so that crew member gets immediately relevant data .  300 * 200 = 60000 rows.

Once we get that manageable subset we may use large data pages to get data for a particular flight when we run the case.

Thoughts ?

August 25, 2016 - 2:40pm


Did you tried using large data page for this particular requirement ? Check out the below link for the same,

August 25, 2016 - 11:23pm
Response to sents

Hi Satish,

     The issue here is how to optimize the data page population itself. The unparameterized master data page has close to 4.4 million records. This has to be brought down to a manageable size before marking it as a large data page.

In such cases it is best to use the operator/user context to retrieve only those relevant records. Then the large data page operations can kick in and make the operation on the data set smoother.

August 26, 2016 - 9:12am

Create a new datapage, and source it from an Activity. In the activity, load data from the original datapage (the huge one) and filter it whatever way you like (e.g. in a Java step). No need to pass any parameters = no problems.

Use that new datapage as a source for your UI controls (lists, forms, etc.). Behind the scenes it will make use of the huge one and filter it.

August 26, 2016 - 10:08am

Sounds like this should work : would this require us to create one datapage for every unique set of params though ?

So if we had (say) two parameters 'name' and 'city' : and one big datapage on the backend which associates names with cities ; and we want to fetch only the rows where 'name=joe' and city='new york'; would we need a datapage for this combination : something like D_MyPage_joe_new_york ?

EDIT: Or does the OOTB 'sync' process already carry a unique identified for the device of some sort  ? If there is such a 'device_id' - we could (with some customization) use that as a 'namespace' for unique params.

So I mean: lets say our device has an ID of 'device123': and this particular device is interested in only the subset of data where 'name=joe, city=new york' - but we are unable to pass the parameters directly in the sync request; so *maybe* we could send up another request to PRPC -uploading our parameters 'name=joe, city=new york' to *another* datapage which indexed by our 'deviceID' 'device123' ; *before* the sync is made.

That way our filtering Activity (presuming this also has some visibility of such a 'deviceid') could look up the parameters that way.

But there is a lot of hypotheticals in that guess above.....


August 29, 2016 - 5:45am
Response to JOHNPW_GCS

Unfortunately, there is no device id sent over sync that can be used for the approach you suggest. 

Best bet in such D_MyPage scenario would be to filter DP by the pxRequestor.pyUserIdentifier == "JOE" assuming Joe is the user.

Otherwise for any random name, you may have to get the entire population down and filter it on client, if the intention is to read and use the data when offline.