Question

Unable to combine two data sets in data flow

Hi All,

We have a requirement where in bulk data from multiple data source will come. We have tried to create a data set uploading a data from csv file. When i tried to combine 2 different data sets using compose for the second data set i am getting error "

This DataSet cannot be used as a secondary Source as it cannot be browsed by keys

I have defined primary key as ID for both first and second data set . I am unable to figure out the way to fix it. Any inputs or help is appreciable.

Thanks in advance,

Ravikiran C

Comments

Keep up to date on this post and subscribe to comments

September 21, 2019 - 8:37am

I think that is expected as your secondary dataset does not support 'browse by keys', see this link: 

https://community.pega.com/sites/default/files/help_v83/procomhelpmain.htm#dsm/tasks/dsm-dataflow-adding-compose-tsk.htm

October 7, 2019 - 1:45am
Response to KevinZheng_GCS

Thank you Kevin for sharing your thoughts. We have decided not to go for Data flows for our requirement as we cant acheive it completly.

September 23, 2019 - 4:43am

The CSV dataset indeed doesnt support that, what it really means that there is no way of indexing the records in such a dataset quickly. The workaround is to copy the CSV dataset into something that can be used for a join/merge, most commonly a DDS (Cassandra) dataset. So you first run a dataflow that copies the entire CSV into a DDS set (with appropriate indices defined), then join/merge that one in your main flow.

October 7, 2019 - 1:45am
Response to Otto_Perdeck

Thank you Otto for sharing your thoughts. We have decided not to go for Data flows for our requirement as we cant acheive it completly.