Guides
Start typing to search…

Kafka (Reading)

Read directly from Kafka

Streamkap supports reading from Kafka.

Customers on a paid plan can enable direct access via Proxy.

Connection details user and topic prefix must be configured for each proxy instance:

  • Kafka brokers example: <tenant-name>.streamkap.net:32400, <tenant-name>.streamkap.net:32401, <tenant-name>.streamkap.net:32402
  • Security protocol: SASL_SSL
  • SASL mechanism: PLAIN
  • username/password
  • topic prefixes and read access permissions

CLI example:


kcat -b some-tenant.streamkap.net:32430 -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X sasl.username=some_user -X sasl.password=***** -C -t my-topic -K \| -c1 -o -1

Confluent's Kafka Python Client example:

from confluent_kafka import Consumer

conf = {
  'bootstrap.servers': 'my-tenant.streamkap.net:32400,my-tenant.streamkap.net:32401,my-tenant.streamkap.net:32402',
  'error_cb': error_callback,
  'security.protocol': 'SASL_SSL',
  'sasl.mechanism': 'PLAIN',
  'sasl.username': 'myuser',
  'sasl.password': os.environ['STREAMKAP_USER_PASSWORD'],
  'client.id': socket.gethostname(),
  # Required to trust AWS root certifiates. Without this we will
  # not be able to successfully coplete the SSL handshake.
  'ssl.ca.location': certifi.where(),
}
c = Consumer(conf)

Integrations supported via Kafka direct read access:

Updated 26 days ago