> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streamkap.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

Change Data Capture using Cloudflare R2 with Streamkap

# Requirements

* R2 Access Key and Secret Access Key with the following permissions to the destination bucket:
  * `Object Read & Write` : Allows the ability to read, write, and list objects in specific buckets.

# Configure R2 Connector

* `Name`: A descriptive name for the connector
* `Access Key`: An Access Key with the appropriate permissions for the bucket to which Streamkap will load data
* `Secret Access Key`: The Secret Access Key with the appropriate permissions for the bucket to which Streamkap will load data
* `Account ID`: Cloudflare Account ID. This is required to access the R2 bucket.
* `Bucket Name`: The name of the bucket to which Streamkap will load data
* `Format`: The format of the file. The following options are available: `CSV`, `JSONL`, `JSON` and `Parquet`
* `Filename Template`: The format of the filename. See below for more information about formatting options.
* `Compression Type`: Compression type for output files. Supported algorithms are `gzip`, `snappy`, `zstd` and `none`. Defaults to `gzip`
* `Output Fields`: List of fields to include in output. Available options are: `key`, `offset`, `timestamp`, `value` and `headers`. Defaults to `value`.

**Filename template** (default: `{{topic}}--{{partition}}--{{start_offset}}`): The format of the filename. You can combine any of the elements below using other text or characters, including dashes (`-`) and underscores (`_`)

| Element                                | Description                                                                                                                                                                                                                                                                                                                                                                                   |
| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `{{topic}}`                            | The Streamkap topic name. For example, a PostgreSQL Source table `web.salesorders` topic's name would be `salesorders`                                                                                                                                                                                                                                                                        |
| `{{partition:padding=true\|false}}`    | The partition number of the records in the file, typically `0`. Streamkap topics and their data can be partitioned for better performance in certain scenarios. For example, a topic `salesorders` has 10 partitions, 0 through to 9. If padding set to true it will set leading zeroes for offset, the default value is false;                                                               |
| `{{start_offset:padding=true\|false}}` | The offset number of the first record in the file. Every record streamed has an incrementing offset number. For example, a topic `salesorders` has 1000 records, offsets 0 through to 999. Note that in the case of a multi-partitioned topic, offset numbers are **not unique** across partitions. If padding set to true it will set leading zeroes for offset, the default value is false; |
| `{{timestamp:unit=yyyy\|MM\|dd\|HH}}`  | The timestamp for when the file was created by the Connector. For example, the template `{{topic}}{{timestamp:unit=yyyy}}-{{timestamp:unit=MM}}` and timestamp of `2024-01-01 20:24` would create a file named `salesorders2024-01`                                                                                                                                                           |
| `{{key}}`                              | The Kafka key                                                                                                                                                                                                                                                                                                                                                                                 |

Currently, supported combinations of variables are:

* topic, partition, start\_offset, and timestamp - grouping by the topic, partition, and timestamp;
* key - grouping by the key.
* key, topic, partition - grouping by the topic, partition, and key.