​
Prerequisites

  • An AWS user with sufficient privileges to create and configure IAM users and roles.

​
AWS Glue Setup

​
1. Create IAM User

It is recommended to create a separate IAM user and role with minimum necessary access.
  • Go to the Amazon IAM Console.
  • Click Users in the sidebar.
  • Click Add users.
  • Enter a User name: e.g., streamkap_user
  • Set the Access type:
    • ✅ Check “Access key – Programmatic access”
    • ❌ Uncheck “Console access” (not needed)
Click Next.
  • Choose Attach policies directly and create a custom policy:
    • Click Create policy (opens in new tab).
    • Go to the JSON tab and paste this policy, ensuring you replace any <...> placeholders as required:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "glue:GetDatabase",
                "glue:GetDatabases",
                "glue:CreateTable",
                "glue:GetTable",
                "glue:GetTables",
                "glue:UpdateTable",
                "glue:DeleteTable",
                "glue:CreatePartition",
                "glue:GetPartition",
                "glue:GetPartitions",
                "glue:BatchCreatePartition",
                "glue:UpdatePartition",
                "glue:DeletePartition",
                "glue:BatchDeletePartition"
            ],
            "Resource": [
                "arn:aws:glue:<region>:<account_id>:catalog",
                "arn:aws:glue:<region>:<account_id>:database/<glue_database_name>",
                "arn:aws:glue:<region>:<account_id>:table/*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket_name>",
                "arn:aws:s3:::<bucket_name>/*"
            ]
        }
    ]
}
For the Connector to access AWS Glue, it needs to be able to assume the user’s role. Please grant the following Trust policy: 
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::300973880807:role/kafkaConnectTenantAccessRole"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

​
Streamkap Setup

​
1. Create the Destination

  • Navigate to Add Connectors.
  • Choose Iceberg.
  • Name: Enter a name for your connector.

​
2. Connection Settings

  • Catalog Type: The type of Iceberg catalog.
    • GLUE:
      • AWS IAM Role: AWS IAM role (e.g., arn:aws:iam:::role/)
  • Region: The AWS region to be used.
  • S3 Bucket Path: Path to the storage location for the Iceberg tables.
  • Schema: An Iceberg table name prefix—equivalent to a database schema (e.g., public, sales, analytics).

​
3. Ingestion Settings

  • Ingestion Mode: Specifies the strategy used to insert events into the Iceberg tables.

    ​
    Changing ingestion mode

    append and upsert modes use different, incompatible methods for loading data into the Iceberg tables. If - for whatever reason - you want to change modes for an existing Iceberg Connector, please create a new Iceberg Destination instead i.e. a separate destination for insert, and for upsert.
    • upsert mode:
      • Primary key fields: Optional. A comma-separated list of field names to use as record identifiers when a primary key’s not present.
Click Save.