> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streamkap.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Apache Iceberg (Generic)

## Prerequisites

* An AWS user with sufficient privileges to create and configure IAM users, roles and S3 buckets.

## Iceberg Data Lake Setup

### 1. Create S3 Bucket

For better **isolation**, **security** and **resource management**, it is recommended to create a new S3 bucket, specifically for your Streamkap-to-Iceberg pipelines.

* Go to the [Amazon S3 Console](https://s3.console.aws.amazon.com/s3).

* Click **Create bucket**.

* Set:

  * **Bucket name:** e.g., `iceberg-bucket`.
  * **Region:** Match your compute region (e.g., `us-west-2`).

* Keep **Object Ownership** set to **ACLs disabled**.

* **Uncheck** “Block all public access” only if you’re managing access via bucket policies (optional).

* Click **Create Bucket.**

Optionally, create a base folder to further organize Iceberg tables created by the connector.

* Open the new bucket.
* Click **Create folder**.
* Enter a name: e.g., `warehouse/streamkap/`.

### 2. Create S3 User

It is recommended to create a separate IAM user and role with minimum necessary access.

* Go to the [Amazon IAM Console](https://console.aws.amazon.com/iam/).

* Click **Users** in the sidebar.

* Click **Add users**.

* Enter a **User name**: e.g., `streamkap_user`

* Set the **Access type**:

  * ✅ Check “Access key – Programmatic access”
  * ❌ Uncheck “Console access” (not needed)

Click **Next**.

* Choose **Attach policies directly** and create a custom policy:

  * Click **Create policy** (opens in new tab).
  * Go to the JSON tab and paste this policy, ensuring you replace any `<...>` placeholders as required:

<CodeGroup>
  ```JSON JSON theme={null}
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "AllowStreamkapAccess",
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::<account_id>:user/streamkap_user"
        },
        "Action": [
          "s3:GetObject",
          "s3:PutObject",
          "s3:DeleteObject",
          "s3:ListBucket"
        ],
        "Resource": [
          "arn:aws:s3:::<bucket_name>",
          "arn:aws:s3:::<bucket_name>/*"
        ]
      }
    ]
  }
  ```
</CodeGroup>

## Streamkap Setup

### 1. Create the Destination

* Navigate to [Add Connectors](https://app.streamkap.com/connectors/add?tab=Destinations).
* Choose **Iceberg**.
* **Name:** Enter a name for your connector.

### 2. Connection Settings

* **Catalog Type:** The type of Iceberg catalog.

  * **`REST`or`HIVE`**:

    * **Catalog Name:** Iceberg catalog name.
    * **Catalog URI:** The Iceberg catalog URI.
    * **AWS Access Key:** The AWS Access Key ID used to connect to S3.
    * **AWS Secret Access Key:** The AWS Secret Access Key used to connect to S3.

* **Region:** The AWS region to be used.

* **S3 Bucket Path:** Path to the storage location for the Iceberg tables.

* **Schema:** An Iceberg table name prefix—equivalent to a database schema (e.g., public, sales, analytics).

### 3. Ingestion Settings

* **Ingestion Mode**: Specifies the strategy used to insert events into the Iceberg tables.

  <Danger>
    **Changing ingestion mode**

    `append` and `upsert` modes use different, incompatible methods for loading data into the Iceberg tables. If - for whatever reason - you want to change modes for an existing Iceberg Connector, please create a **new** Iceberg Destination instead i.e. a separate destination for `insert`, and for `upsert`.
  </Danger>

  * **`upsert`mode**:
    * **Primary key fields:** Optional. A comma-separated list of field names to use as record identifiers when a primary key's not present.

Click **Save**.