Create New Destination

The maximum number of records returned in a single call to poll()

The maximum number of active tasks.

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Upload the BigQuery service-account JSON key file.

Name of the destination BigQuery dataset. The dataset must already exist.

Time partitioning granularity used when auto-creating tables. Select NONE to create non-partitioned tables.

Optional record field to partition by. Leave blank for ingestion-time partitioning.

Optional comma-separated list of fields to cluster the table by (max 4).

Optional. Automatically delete partitions older than this many days from tables this destination creates. Leave blank to keep all partitions. Ignored when Time Partitioning is NONE.

Automatically create BigQuery tables for new topics.

Allow adding new columns to BigQuery tables when the schema evolves.

Allow changing BigQuery columns from REQUIRED to NULLABLE when the schema evolves.

The maximum number of records returned in a single call to poll()

The maximum number of active task

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Upsert or append modes are available

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database (applies to upsert only)

ClickHouse Hostname Or IP address

Username to access ClickHouse

Password to access the ClickHouse

ClickHouse Port. For example, 8443

ClickHouse database

Enable TLS for network connections

Per topic configuration in JSON format

Allow JSON data type in ClickHouse, make sure the ClickHouse server supports it. If not set, the connector will use String type for JSON data.

Optional prefix prepended to every destination table name (e.g. 'prod_' produces tables like 'prod_customers'). Only letters, digits, and underscores are allowed.

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

CockroachDB Hostname

CockroachDB Port. For example, 26257

Database name

Username to access the database

Password to access with the database

Schema for the associated table name

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Specifies the strategy used to insert events into the database

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Specifies how the connector resolves the primary key columns from the event

Optional. Either the name of the primary key column or a comma-separated list of fields to derive the primary key from.

Falls back to Streamkap's default for tables where no match is found

Map source tables to specific destination tables. Input should be the format of source_table_name:destination_table_name separated by a new line

The maximum number of active tasks. NOTE: Increasing this value may increase parallelism and throughput but can also lead to higher costs on databricks side.

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Upsert or append modes are available

Token

JDBC URL

Connection Timeout

The name of the Databricks catalog to use.

Partition tables or not

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Schema for the associated table name

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Controls how long the connector waits to accumulate a larger batch before writing to Databricks. Higher values = larger batches = better throughput and fewer MERGE INTOs, but higher latency.

Falls back to Streamkap's default for tables where no match is found

Map source tables to specific destination tables. Input should be the format of source_table_name:destination_table_name separated by a new line

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Db2 Hostname

Db2 Port. For example, 50000

Database name

Username to access the database

Password to access with the database

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Specifies the strategy used to insert events into the database

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Specifies how the connector resolves the primary key columns from the event

Optional. Either the name of the primary key column or a comma-separated list of fields to derive the primary key from.

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

GCP credentials as a JSON string with escaped quotes (" into "). \n in the private_key field should also be escaped as \n. Example format: {"type": "service_account","project_id": "XXXXXX", ...}

The GCS Bucket to use.

The format to use when writing data to the store.

The format of the filename. See documentation for more information about formatting options.

Prefix for the filename. Prefixes can be used to specify a directory for the file (e.g. dir1/dir2/).

Compression type for files written to GCS.

A comma separated list of fields to include in output? Options to include key, offset, timestamp, value, headers.

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

The URL to send data to.

The HTTP authorization type.

Static content of Authorization header (required for STATIC authorization).

URL for fetching OAuth2 access token.

OAuth2 client id.

OAuth2 client secret.

OAuth2 client scope.

Value of Content-Type header for each request.

Additional headers in header:value format, comma-separated.

Proxy hostname.

Proxy port.

Enable batching multiple records in a single HTTP request.

Maximum size of a record batch (1 to 1,000,000).

Enable buffering and only send records when batch.max.size reaches or batch.max.time.ms hits.

Maximum time to wait before flushing a batch (1 to 1,000,000,000 ms).

Prefix for record batches.

Suffix for record batches.

Separator for records in a batch.

Maximum number of retries on errors.

Time in ms to wait before retrying.

HTTP response timeout in seconds.

Format for serializing decimals: BASE64 (default) or NUMERIC.

Error tolerance setting. Note: Cannot be used when Batching is enabled.

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Type of Iceberg catalog

How to authenticate with the Iceberg catalog. 'No Auth' for catalogs that don't require authentication. 'Token' for bearer token authentication (e.g., Cloudflare R2). 'OAuth2 Credential' for OAuth2 client credential authentication (e.g., Polaris, DataHub).

Iceberg catalog name

AWS IAM role (e.g., arn:aws:iam::<your-account>:role/<role-name>)

Catalog endpoint URL. For REST catalogs use https://… ; for Hive Metastore use thrift://… ; AWS Glue uses the regional endpoint.

Bearer token used for catalog authentication.

OAuth2 client credential in client_id:client_secret format.

OAuth2 scope for catalog authentication (e.g., PRINCIPAL_ROLE:ALL).

OAuth2 token endpoint URI. Required when the REST catalog endpoint is not also the OAuth2 authorization server.

Optional OAuth2 audience parameter to request from the token endpoint.

Optional OAuth2 resource parameter to request from the token endpoint.

Enable to provide your own S3 access key and secret. Disable if the catalog vends storage credentials automatically.

The access key ID used to connect to S3 or S3-compatible storage.

The secret access key used to connect to S3 or S3-compatible storage.

The AWS region to be used. For Cloudflare R2, select 'auto'.

The Iceberg warehouse identifier. For R2 Data Catalog, this should be the warehouse name configured in the catalog. For S3-based catalogs, this is typically an S3 or S3-compatible path (e.g., s3://bucket/path).

Name of the database schema that contains the table (e.g., public, sales, analytics).

Specifies the strategy used to insert events into the database

Optional. A comma-separated list of field names to use as record identifiers when key fields are not present in Kafka messages

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database (applies to upsert only)

Enable lightweight compaction of small data files after commits

Target file size in bytes for compaction. Files smaller than this are candidates for merging

Minimum number of small files required before triggering compaction

Maximum number of small files to compact in a single run

Number of Iceberg commits between compaction checks

Expire old snapshots and delete unreferenced data files after compaction

Number of most recent snapshots to retain when expiring snapshots after compaction

The maximum number of records returned in a single call to poll()

Maximum number of tasks for the connector

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

A comma-separated list of host and port pairs that are the addresses of the Destination Kafka brokers. This list should be in the form host1:port1,host2:port2,...

The format to use when writing data to kafka

Include schema in json message

Destination kafka schema registry url

Prefix for destination topics

Suffix for destination topics

If Kafka destination is used to sync data between services or to apply some transformations, it would be better to rename this field to _streamkap_offset_sync or _streamkap_offset_transform to avoid conflicts with the destination on the other service

Add the original topic name to the output record. 'off' disables it, 'header' adds it as a Kafka header, 'field' adds it as a _streamkap_topic column in the destination table.

Password for setting up the Kafka proxy

A comma-separated list of IPs/CIDRs

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Upsert or append modes are available

Motherduck Token

The name of the Motherduck database/catalog to use.

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Schema for the associated table name

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Falls back to Streamkap's default for tables where no match is found

Map source tables to specific destination tables. Input should be the format of source_table_name:destination_table_name separated by a new line

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

MySQL Hostname

MySQL Port. For example, 3306

Database name

Username to access the database

Password to access with the database

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Specifies the strategy used to insert events into the database

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Specifies how the connector resolves the primary key columns from the event

Optional. Either the name of the primary key column or a comma-separated list of fields to derive the primary key from.

Falls back to Streamkap's default for tables where no match is found

Map source tables to specific destination tables. Input should be the format of source_table_name:destination_table_name separated by a new line

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Oracle Hostname

Oracle Port. For example, 1521

Database name

Username to access the database

Password to access with the database

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Specifies the strategy used to insert events into the database

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Specifies how the connector resolves the primary key columns from the event

Optional. Either the name of the primary key column or a comma-separated list of fields to derive the primary key from.

Falls back to Streamkap's default for tables where no match is found

Map source tables to specific destination tables. Input should be the format of source_table_name:destination_table_name separated by a new line

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

API key for authenticating with Pinecone

Name of the target Pinecone index

Proxy host for Pinecone connections

Proxy port for Pinecone connections (required if Proxy Host is set)

Mapping between Kafka topic and Pinecone namespace. Use ${topic} for dynamic topic name substitution

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created namespaces, set to NONE

Strategy to generate document IDs. 'None' uses random UUIDs (append-only, no upsert/delete support)

Field name containing the ID in Kafka record

Field name containing the embedding vector

Treat null record values or __deleted=true as deletes. Requires Document ID Strategy to be set to 'Kafka ID' (not 'None')

Number of records per batch

Maximum number of retry attempts for failed operations

Backoff time between retries in milliseconds

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

PostgreSQL Hostname

PostgreSQL Port. For example, 5432

Database name

Username to access the database

Password to access with the database

<span>Streamkap will connect to SSH server in your network which has access to your database. This is necessary if Streamkap cannot connect directly to your database. <a href='https://docs.streamkap.com/streamkap-ip-addresses#streamkap-ip-addresses' class='docs-url' target='_blank'>View the Streamkap IP addresses to allowlist on your SSH server</a> </span>

Hostname of your SSH server

Port of your SSH server

User that allows Streamkap to connect to SSH server

Schema for the associated table name

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Specifies the strategy used to insert events into the database

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Specifies how the connector resolves the primary key columns from the event

Optional. Either the name of the primary key column or a comma-separated list of fields to derive the primary key from.

Rename destination tables using a regex pattern. The regex is matched against the full topic name — use capture groups to extract parts. Example: <code>..(.)</code> extracts the table name from <code>source_123.public.orders</code>. Leave empty to skip.

The replacement value for the matched regex. Use <code>$1</code>, <code>$2</code> etc. to reference capture groups from the pattern. Example: <code>$1</code> with the pattern above produces <code>orders</code>.

Optional second rename rule, applied after the first. Use when a single regex is not enough. Leave empty to skip.

The replacement value for the second rename rule. Use <code>$1</code>, <code>$2</code> etc. to reference capture groups.

Falls back to Streamkap's default for tables where no match is found

Map source tables to specific destination tables. Input should be the format of source_table_name:destination_table_name separated by a new line

Public key to add to SSH server

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Cloudflare R2 account ID. This is the account ID you see in the Cloudflare dashboard.

The Access Key ID used to connect to R2.

The Secret Access Key used to connect to R2.

The R2 Bucket to use.

The format to use when writing data to the store.

The format of the filename. See documentation for more information about formatting options.

Prefix for the filename. Prefixes can be used to specify a directory for the file (e.g. dir1/dir2/).

Compression type for files written to S3.

A comma separated list of fields to include in output? Options to include key, offset, timestamp, value, headers.

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Redis Hostname

Redis Port. For example, 6379

Username to access the Redis database

Password to access the Redis database

Enable TLS for network connections

Redis key to stream the data

Data Type of Redis Key. At the moment, only Stream and List are supported

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

Domain of the leader of the cluster

Port number for incoming connections to the leader

Name of the database on the cluster

Username to access the database

Password to access with the database

List of comma-separated primary key field names

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Schema for the associated table name

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

How Streamkap authenticates with S3. Choose Access Keys to use an AWS Access Key ID and Secret. Choose Cross-Account Role to have Streamkap assume an IAM role in your account using a Role ARN and External ID (recommended for production — see https://docs.streamkap.com/aws-cross-account-iam).

The AWS Access Key ID used to connect to S3.

The AWS Secret Access Key used to connect to S3.

ARN of the IAM role Streamkap should assume to write to your S3 bucket (e.g. arn:aws:iam::<your-account-id>:role/<role-name>). The role's trust policy must allow the Streamkap account to assume it.

The External ID configured in your role's trust-policy condition. Streamkap passes this value verbatim when assuming the role.

The AWS region to be used

The S3 Bucket to use.

The format to use when writing data to the store.

The format of the filename. Static text can be added anywhere in the template, including at the start, to act as a directory prefix (e.g. dir1/dir2/{{topic}}-{{partition}}-{{start_offset}}). See documentation for more information about formatting options.

Compression type for files written to S3.

Maximum number of records buffered into a single file before it is rotated. The rotated file is still only uploaded to S3 on the next flush interval. Set to 0 for unlimited (one file per topic-partition per flush interval).

Size of each part in a multipart upload to S3, in bytes. Larger parts mean fewer S3 API calls for big files but more memory used per upload. AWS requires multipart parts to be at least 5MB (except the last part of a file).

When enabled (default), each output record is wrapped in an envelope with Kafka metadata (key, offset, timestamp, headers) alongside the value. Disable to write only the record's own value structure. No effect on CSV; for Parquet, only applies when the value is a record or map.

A comma separated list of fields to include in output? Options to include key, offset, timestamp, value, headers.

The maximum number of records returned in a single call to poll()

The maximum number of active tasks

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

SQL Server Hostname

SQL Server Port. For example, 1433

Database name

Username to access the database

Password to access with the database

Schema for the associated table name

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Specifies the strategy used to insert events into the database

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database

Specifies how the connector resolves the primary key columns from the event

Optional. Either the name of the primary key column or a comma-separated list of fields to derive the primary key from.

Falls back to Streamkap's default for tables where no match is found

Map source tables to specific destination tables. Input should be the format of source_table_name:destination_table_name separated by a new line

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'

List of regex patterns to search for. Comma separated list in format 'regex1,regex2'

List of replacement values for each regex pattern. Comma separated list in format 'regex1,regex2'

Truncate or nullify oversized string fields. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*'). WARNING: Do not include primary key columns - truncation/nullification could cause data loss or failures.

Columns to exclude from oversized records processing. Comma separated list in format 'table1.column1,table2.column2'.

Maximum allowed byte size per field. Fields exceeding this size will be truncated or nullified. Required when using Oversized Records transform.

Action for oversized fields: TRUNCATE (trim to max size) or NULLIFY (set to null).

Suffix to append to truncated values (e.g., '...[TRUNCATED]'). Leave empty for no suffix.

Optional overall record size limit in bytes. Records are NOT dropped - only a warning is logged if record exceeds this after field processing. Set to -1 to disable.

Column data types that should never be truncated. Comma-separated. Defaults exclude JSON and XML columns.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Mask (anonymise) string column(s) in-place. Comma separated list of table columns in format 'table1.column1,table2.column2'. Supports wildcards (e.g., 'mytable.*').

Columns to exclude from masking. Comma separated list in format 'table1.column1,table2.column2'.

Masking algorithm. Hash functions are deterministic (same input -> same token) and length-preserving.

Secret salt prepended before hashing (used by the SHA256/MD5 functions). Strongly recommended: without it, low-cardinality values (phone, SSN, email) are reversible via a precomputed rainbow table.

Fill character used by the REDACT function (length is preserved).

Constant replacement used by the FIXED function.

Whether null fields should use schema default values. Set to false to preserve user-set NULLs from source.

Warning: Should only be used in conjunction with numeric conversion or other conversions. If field remained string after previous conversion, rename to field to <previous-field-name>_str. Comma separated list of table columns in format 'table1.column1,table2.column2'

Regular expression for matching topic name parts to use as the destination table (database) or file (file storage) name

JSON mapping of source to target column names. Keys should be schema.table.column, table.column or just column. Values must be valid column names (no dots or spaces).

Example: { "public.orders.amount": "Amount", "orders.quantity": "Qty", "customer_id": "CustomerId" }

Drop columns. Comma separated list of table columns to drop in format 'table1.column1,table2.column2'

When enabled, converts ALL optional columns to required (NOT NULL). The exclude list is still respected. Enabling this automatically disables 'Mark Column(s) as Optional'.

Controls how NULL values are handled when columns become required. NONE: throws an error on NULL values. TYPE_MIN: replaces NULLs with type-specific sentinel values (e.g. MIN_VALUE for integers, 'null' for strings, epoch for dates).

Mark columns as optional/nullable. Comma separated list of table columns in format 'table1.column1,table2.column2'

Comma-separated list of field mappings e.g. <code>field1:newField1,field2:newField2</code>.

Headers to columns. Comma separated list of headers using format <code><header name>:<header type>[:field name]</code> e.g. 'headerKey1:STRING,headerKey2:INT32:customKey2Name'

The URL for accessing your Snowflake account. This URL must include your account identifier. Note that the protocol (https://) and port number are optional.

User login name for the Snowflake account.

The private key to authenticate the user. Include only the key, not the header or footer. If the key is split across multiple lines, remove the line breaks.

If checked (default), provide your SSH key's passphrase, otherwise, uncheck for SSH keys without passphrase.

The passphrase is used to decrypt the private key.

The name of the snowflake warehouse.

The name of the database that contains the table to insert rows into.

The name of the schema that contains the table to insert rows into.

Automatically generates a Snowflake schema if it does not already exist.

The name of an existing role with necessary privileges (for Streamkap) assigned to the <Username>

<span>Upsert or append modes are available. NOTE: when switching append to upsert, existing data must be deduplicated or deleted. <a href='https://docs.streamkap.com/snowflake#upsert-mode' class='docs-url' target='_blank'>Read more about upsert mode</a> </span>

Specifies whether the connector processes DELETE or tombstone events and removes the corresponding row from the database (applies to upsert only)

Controls how schema evolution is handled by the sink connector. For pipelines with pre-created destination tables, set to NONE

Specifies whether the connector should create Hybrid Tables (applies to upsert only)

Specifies whether the connector should create Dynamic Tables & Cleanup Tasks (applies to append only)

These template queries run for each table the first time a record is streamed for them.

Can be used as <code>{{dynamicTableName}}</code> in dynamic table creation SQL. It can use input JSON data for more complex mappings and logic.

Use <code>{"TABLE_DATA": {"{table_name}": {"{key}": "{value}"}, ...}, ...}</code> to set table specific data. This data will be available in the custom SQL templates e.g. <code>SELECT {{key}}</code>.

Mapping between the tables that store append-only data and the deduplicated tables. The dedupeTable in mapping will be used for QA scripts. If dedupeSchema is not specified, the deduplicated table will be created in the same schema as the raw table.

Define custom topic-to-table name mapping using regex. Format: <code>matching_pattern:replacement_pattern</code>. Use $1, $2, etc. for captured groups. Example: <code>^([-\w]+.)([-\w]+.)?([-\w]+.)?([-\w]+.)?([-\w]+):$5</code> uses only the last segment as table name.

Comma-separated list of key fields to copy into each record's value so they land as columns at the destination. Use * to copy all key fields, or <topic>.<field> patterns to scope by topic and field name (e.g. *.pk,*.sk where * matches any topic). Leave empty to disable.

The maximum number of records returned in a single call to poll()

When enabled, preserves NULL values from the source database instead of replacing them with schema default values. Enable this if you need to distinguish between explicit NULLs and default values.

Whether to quote identifiers in SQL statements

Convert column(s) to Integer - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Float - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to Decimal - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

If true, truncate decimals to a 38 precision

Convert column(s) to String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert column(s) to JSON String - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to JSON String? - if possible.

Convert column(s) to Binary JSON - if possible. Comma separated list of table columns in format 'table1.column1,table2.column2'

Convert all Structs and Arrays to Binary JSON? - if possible.

Converts all JSON String column(s) to Binary JSON - if possible.

Replaces column(s) value with a user-defined string. Comma separated list of table columns in format 'table1.column1,table2.column2'