Skip to main content
Version: v2

Connecting to EIMP Entity Change Kafka Topic

Once access is provisioned, owners and viewers of the subscription will receive an email confirming the Kafka topic subscription. This email will include a link to download the certificate required for connecting to the Kafka topic. * Note:* The link expires 168 hours (7 days) after it is provisioned.

Extracting Certificate Information​

The certificate .zip folder will contain the following files:

  • ca.crt
  • cert.pem
  • jks_passwords.txt
  • key.pem
  • keystore.jks
  • truststore.jks

For Kafka integration, you will only need the following files:

  • keystore.jks
  • truststore.jks
  • jks_passwords.txt

Recommendations​

  1. Update .jks Passwords: It is recommended to update the passwords for the .jks files.
  2. Convert .jks to Base64: Convert the .jks files to Base64 format before storing them in the Azure Key Vault.

Validating the Certificate​

Before converting the .jks files to Base64, validate the certificate using the following command:

keytool -list -keystore keystore.jks -storepass <password>

Update Keyvault Secrets​

Once that the .jks have been converted to base64 strings, then we can update the keyvaults in Azure. We have two different vaults, depending on the environment. In non-prod, EIMP entity change kafka topic subscription is shared across all environments.

Non-prod

  • kafka-ssl-password-dev
  • truststore-b64-dev
  • keystore-b64-dev
  • kafka-group-id
  • kafka-bootstrap-servers
  • kafka-schema-registry-url

Prod

  • kafka-ssl-password
  • truststore-b64
  • keystore-b64
  • kafka-group-id
  • kafka-bootstrap-servers
  • kafka-schema-registry-url

Some of these properties are retrieved from the HCP Data Catalog subscription information, for instance, the group id, boostrap servers and the schema registry URL.

Test Kafka connection locally​

From our local machines, we have two options, spinning up a Kafka instance using Docker, or else connecting directly to the Kafka topic by providing the necessary information.