Question

Real time dataflow with Kafka not distributed

Realtime data flow does not get distributed to all real time nodes .

Starting Kafka Data set has Partitions set

scoringPartitionKey is set in the topic message the real time data flow is listening to.

{....."scoringPartitionKey":107...........}

appreciate if someone could share details of how to set up the data flow to run on all Realtime nodes.

***Edited by Moderator: Lochan to update platform capability tags***

***Edited by Moderator Marissa to update SR Details***

Group Tags

Comments

Keep up to date on this post and subscribe to comments

September 20, 2019 - 6:53pm

Sree,

How many partitions are there in your external kafka topic? Looks like only 1. Can you confirm?

 

September 23, 2019 - 10:39am

Kevin, nice from hear from you.  The number of partitions was only 1 . we increased to 50 and now the load gets distributed to all 12 nodes . Thank you