Requirements

Configure S3 Connector

Name: A descriptive name for the connector
AWS Access Key: An Access Key with the appropriate permissions for the bucket to which Streamkap will load data
AWS Secret Access Key: The Secret Access Key with the appropriate permissions for the bucket to which Streamkap will load data
Region: Name of the region for bucket to which Streamkap will load data
Bucket Name: The name of the bucket to which Streamkap will load data
Format: The format of the file. The following options are available: Avro, Byte Array, JSON and Parquet
Directory: The name of the directory to which Streamkap will load data

If the Format Type selected is JSON or Byte Array:

Compression Type: How Streamkap should compress the files. The following options are available: None or gzip

If the Format Type selected is Avro:

Avro Compression Type: How Streamkap should compress the files. The following options are available: null, deflate, snappy, or bzip2

If the Format Type selected is Parquet:

Parquet Compression Type: How Streamkap should compress the files. The following options are available: None, gzip, snappy, lz4, brotli, zstd, or lzo
Partition Field: How the files should be partitioned. Automatic will partition the data automatically and not based on any particular field. Field will partition the data by the values of a specified field. Time will partition the data by the time of a particular field

If the Partition Field is set to Time:

Time Partition Directory Path Format: The format of the partitioned directories. For example, if you set format to'year'=YYYY/'month'=MM/'day'=dd/'hour'=HH, the data directories will have the format /year=2015/month=12/day=07/hour=15/
Time Partition Timezone: The timezone of the partitioned directories. Accepts short and long standard names like: UTC, PST, ECT, Europe/Berlin, Europe/Helsinki, or America/New_York