---
title: "Google Cloud Storage"
description: "Stream change data capture records into a Google Cloud Storage bucket from Streamkap, with JSON Lines, JSON Array, or Parquet formats."
---

## Requirements

* A Google Cloud Storage bucket for Streamkap to write to.
* A Google Cloud service account with a JSON key. The service account needs these permissions on the bucket:

  * `storage.objects.create`
  * `storage.objects.delete` (needed to overwrite files)

  The predefined **Storage Object User** role (`roles/storage.objectUser`) includes both. You can also create a custom role with only these two permissions.

## Configure Google Cloud Storage Connector

* `Name`: A descriptive name for the connector
* `GCS Credentials JSON Service Account Key`: The service account's JSON key. Paste it as a single line, with every double quote escaped as `\"` and every `\n` inside `private_key` escaped as `\\n`. For example: `{\"type\": \"service_account\", \"project_id\": \"my-project\", ...}`
* `Bucket Name`: The name of the bucket to which Streamkap will load data
* `Format`: The format of the file. Choose `JSON Lines`, `JSON Array` or `Parquet`

:::info
CSV output is not currently supported, even though `CSV` appears in the `Format` list. Writing CSV requires reading the source data as raw, unparsed bytes (a `ByteArrayConverter`) instead of as structured records, and Streamkap doesn't currently offer that as a serialization option. Select one of the other formats. If you have a use case that needs CSV output, contact [Streamkap support](mailto:support@streamkap.com) to discuss options.
:::

* `Filename Template`: The format of the filename. A file extension is appended automatically based on the selected `Format` and `Compression Type`, so do not include one in the template. See below for more information about formatting options.
* `Filename Prefix`: Text added to the start of every filename. Use it to write files into a folder in the bucket, for example `dir1/dir2/`.
* `Compression Type`: Compression type for output files. Supported algorithms are `gzip`, `snappy`, `zstd` and `none`. Defaults to `gzip`.
* `Output Fields`: List of fields to include in output. Available options are: `key`, `offset`, `timestamp`, `value` and `headers`. Defaults to `value`.

Streamkap adds a `_streamkap_offset` field to each record's value. It holds the record's position in its Streamkap topic.

**Filename template** (default: `{{topic}}-{{partition}}-{{start_offset}}`): The format of the filename, excluding the extension. You can combine any of the elements below using other text or characters, including dashes (`-`) and underscores (`_`).

| Element                                | Description                                                                                                                                                                                                                                                                                       |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `{{topic}}`                            | The Streamkap topic name. For example, a PostgreSQL Source table `web.salesorders` topic's name would be `salesorders`                                                                                                                                                                            |
| `{{partition:padding=true\|false}}`    | The partition number of the records in the file, typically `0`. Streamkap topics and their data can be partitioned for better performance in certain scenarios. For example, a topic `salesorders` has 10 partitions, 0 through to 9. If padding is set to true, the number has leading zeroes. Defaults to false |
| `{{start_offset:padding=true\|false}}` | The offset number of the first record in the file. Every record streamed has an incrementing offset number. Offset numbers are **not unique** across partitions of the same topic. If padding is set to true, the number has leading zeroes. Defaults to false                                    |
| `{{timestamp:unit=yyyy\|MM\|dd\|HH}}`  | The timestamp for when the file was created by the connector. For example, the template `{{topic}}{{timestamp:unit=yyyy}}-{{timestamp:unit=MM}}` and a timestamp of `2024-01-01 20:24` would create a file named `salesorders2024-01`                                                          |
| `{{key}}`                              | The Kafka key                                                                                                                                                                                                                                                                                     |

Currently, supported combinations of variables are:

* topic, partition, `start_offset`, and timestamp - grouping by the topic, partition, and timestamp;
* key - grouping by the key.
* key, topic, partition - grouping by the topic, partition, and key.