Transforms overview

Filter, enrich, join, roll up, and fan out streaming records with Streamkap pipeline transforms written in JavaScript, Python, or SQL.

Transform Types

Below are the current and upcoming transform types Streamkap offers for data transformation:

Transform Name	Description	Supported Languages	Status
`Transform / Filter Records`	Apply custom logic to modify or filter records.	JavaScript	✅ Available
`Enrich`	Lookup values from a cached topic to enrich records. Handles schema evolution and pass-through fields.	SQL	✅ Available
`Join`	Join multiple topics. Supports schema evolution and pass-through fields.	SQL	✅ Available
`Enrich (Async)`	Asynchronously call REST APIs to enrich incoming records using dynamic payloads.	JavaScript, Python	✅ Available
`Rollup`	Perform in-stream aggregations using familiar SQL `GROUP BY` syntax. Correlate multiple CDC events for the same entity, aggregating by primary key.	SQL	✅ Available
`Un Nesting`	Break nested objects or arrays into separate topics. Ideal for unrolling deeply nested documents.	JavaScript, Python	🛠️ Coming Soon
`Fan Out`	Route a single record to multiple output topics based on custom logic.	JavaScript, Python	✅ Available
`Topic Router`	Merge, copy, or route topics via the Kafka destination connector with byte pass-through. Rewrites topic names and Avro schema IDs without value deserialization.	—	✅ Available
`Toast Handling`	Replace PostgreSQL `TOAST` placeholders with cached, encrypted original values to avoid costly re-selects.	TBC	🛠️ Coming Soon

Professional development environments and starter kits for building transforms:

Tool Name	Description	Transform Types Supported	Status
`TypeScript Development Kit`	Professional development environment with TypeScript, npm dependencies, comprehensive testing, and production bundling.	Map/Filter, Fan Out, Enrich (Async), Un-nesting	✅ Available

The TypeScript development kit covers authoring, testing, and deploying a transform locally, and data types lists what a transform can read and emit.