Question

Data-Admin-Kafka Configuration with Pega 8.3 Cloud

We are looking to Integrate Kafka with Pega 8.3.0 Version (Cloud). Steps that we have taken so far are as follows:

1) Created a Kafka Data Instance and provided the Host Name and Port Number. We are able to telnet successfully to this Host name and port.

2) In the Security Setting: Use SSL Configuration and created a Keystore instance which is referred in Kafka configuration. Also, checked Use Client Certificate and created another keystore instance which is referred in Kafka configuration.

3) Tried to save the instance and got an error
'Connection failed Error connecting to Kafka. Timeout expired while fetching topic metadata'

We have following questions:
1) How do we enable the JAAS configuration in the Kafka rule. (As per pega help, please find the screen shot, but will we be able to do if we are using Pega Cloud Instance). I think this configuration is possible only if we are using on-prem. What we can do to get this?

Please guide us to support team who can help us in this regard, or if any one else have worked with Pega cloud to resolve this.

Comments

Keep up to date on this post and subscribe to comments

November 20, 2019 - 7:40am

I assume your Kafka is in your old network. Please engage with PegaCloud support to check if certain networking ports required by Kafka are configured through vpn tunnel correctly.

November 22, 2019 - 4:08am

We have been asked  to raise a change request on cloud service to perform following changes.

Creating a ANYNAME.conf file which should have the KafkaClient user name and password in following format.

KafkaClient { org.apache.kafka.common.security.plain.PlainLoginModule required

username= "*******"

password = "*******";

}

Step 0: Shut down App tier nodes.

Step 1: After creation of ANYNAME.conf, place it into the server directory and note down the path

Step 2: add below JVM argument for Apptier nodes:
Djava.security.auth.login.config = /<path>/ANYNAME.conf

Step 4: Bring up App tier nodes.

Check Kafka connectivity. We are still awaiting our change request to be addressed by Pega Cloud support. Once these changes are done, we can post any further updates here.

November 29, 2019 - 4:29am

After implementation of above change request on Pega Cloud, we are still experiencing issues in producing and consuming messages to/from Kafka Topic.

One error i.e. frequently showing up in the log file is:

"Waking up the sender since topic NewTopicOnSSL partition 0 is either full or getting a new batch"

Not sure whether this error is something for which we require to fix from Pega end or Kafka host.

Any suggestions?