Documentation Index
Fetch the complete documentation index at: https://docs.streamkap.com/llms.txt
Use this file to discover all available pages before exploring further.
Overview
The Shopify Webhook source receives events from your Shopify store at a dedicated HTTPS endpoint and routes each event to a Kafka topic based on its resource type (orders, products, customers, draft orders, fulfillments, inventory items, collections, etc.).
You point Shopify at the Streamkap-generated webhook URL, register the topics you care about, and Streamkap takes care of parsing, key extraction, schema inference, and routing. For initial loads or ad-hoc backfills, the source can also snapshot data directly from the Shopify GraphQL Admin API.
This connector is in Beta. Behaviors and defaults may change before general availability.
Prerequisites
- A Shopify store with Admin access (required to install apps and register webhooks).
- A Shopify Partners account so you can create a Dev Dashboard app for OAuth credentials (recommended) — or an existing legacy custom app with a permanent access token.
- A Streamkap workspace with permission to create source connectors.
- A clear list of the Shopify resources you want to capture (orders, products, customers, etc.).
How It Works
- Endpoint provisioning — When you create the source, Streamkap generates a unique HTTPS webhook URL and an API key.
- Shopify subscriptions — In Shopify, you register webhook subscriptions for the topics you want (
orders/create, products/update, …) pointing at the Streamkap URL with the API key passed as a query parameter.
- Header-based routing — Each incoming request carries an
X-Shopify-Topic header (orders/create, customers/delete, etc.). The Shopify payload router maps the resource segment to a topic — orders/* events go to orders, products/* events go to products, and so on.
- Key extraction — The router pulls the resource
id from the JSON body and uses it as the Kafka message key, enabling upsert-style consumption downstream.
- HMAC verification (optional) — When you provide the Shopify app’s client secret, each payload is verified against the
X-Shopify-Hmac-Sha256 header. Invalid payloads are rejected and (if configured) routed to the DLQ.
- Optional flattening / fan-out — Nested fields can be promoted to top level, and array fields (line items, variants, addresses) can be fanned out into their own topics.
- Optional snapshot — For initial loads or ad-hoc backfills, the source queries the Shopify GraphQL Admin API with cursor pagination and emits records alongside the live webhook stream.
Streamkap Setup
1. Create the Source
- Navigate to Sources and choose Shopify.
- Give the source a memorable Name (for example,
shopify-prod).
2. Connection Settings (Auth tab)
| Field | Required | Description |
|---|
| Webhook URL | Auto | Read-only. Endpoint that Shopify posts events to. Generated on save. Append ?api_key=<API_KEY> when registering webhooks (see Shopify Setup). |
| API Key | Auto | Read-only and encrypted. Sent by Shopify as the api_key query parameter to authenticate each request. Generated on save. |
| Store URL | Yes (for snapshots) | Your store URL — for example https://yourstore.myshopify.com. Required for snapshot mode. |
| Client ID | Conditional | Client ID from your Shopify Dev Dashboard app. Use together with Client Secret for the recommended client-credentials flow (tokens auto-refresh every 24 hours). |
| Client Secret | Conditional | Client Secret from your Shopify Dev Dashboard app. Encrypted at rest. |
| Access Token (Legacy) | Conditional | Static access token from a legacy custom app. Provide either Access Token or Client ID + Client Secret. |
| API Version | No | Shopify Admin API version used by snapshots. Default 2024-10. |
| HMAC Secret | No | Your Shopify app’s client secret. When set, every webhook is verified against the X-Shopify-Hmac-Sha256 header. Leave empty to skip verification. |
3. Schema (Shopify Resources)
In the Schema tab, choose the resources you want to capture. The default is orders,products,customers.
A resource is the first segment of the X-Shopify-Topic header — for example orders/create and orders/cancelled both map to the orders resource. Each resource becomes a Kafka topic of the same name.
The dropdown is pre-loaded with the resources surfaced in the Shopify Admin UI’s webhook picker (Settings → Notifications → Webhooks → Create webhook):
| Category | Resources |
|---|
| Core entities | orders, products, customers, draft_orders, fulfillments, inventory_items, inventory_levels, collections, shop |
| Customer events | customer (for tag-change events), customer_groups, customer_account_settings |
| Order lifecycle | order_transactions, refunds, fulfillment_orders, fulfillment_holds |
| Storefront | discounts, carts, checkouts |
| Inventory operations | inventory_shipments, inventory_transfers |
| Store configuration | locations, markets, themes, tender_transactions |
Any fan-out topics you configure (see Fan-out) are added automatically.
Advanced — GraphQL Admin API registration. If you register webhooks via the GraphQL Admin API instead of the Admin UI, additional WebhookSubscriptionTopic values are available (for example SUBSCRIPTION_CONTRACTS_*, COMPANIES_*, METAOBJECTS_*, BULK_OPERATIONS_FINISH). The Schema field accepts any resource name — just type the first-segment of the topic header and Streamkap will route it.
Snapshot support is narrower than webhook routing. Only orders, products, customers, draft_orders, collections, and inventory_items can be snapshotted via the GraphQL Admin API — see Snapshot. All other resources stream via webhooks only.
4. Settings
| Field | Default | Description |
|---|
| Unselected Resource Behavior | DEFAULT_TOPIC | What to do when an event arrives for a resource not in your Schema list. DEFAULT_TOPIC, SKIP, or FAIL. |
| Default Topic for Unselected Resources | unknown | Topic used when behavior is DEFAULT_TOPIC. Only shown when the option above is set to DEFAULT_TOPIC. |
| Flatten Record Fields | true | Promote nested fields to top level. Recommended for most warehouse destinations. |
| Flatten Prefix | (empty) | Prefix for flattened fields. Empty means Shopify field names are used directly (id, email, total_price, …). Only shown when flattening is enabled. |
| Include Webhook Metadata | false | Keep Shopify webhook headers (_shop_domain, _event_id, _triggered_at, _api_version, _webhook_id) on the output record. Disable for upsert / state-table mode; enable for audit-log mode. |
| Fan-out Fields (advanced) | (empty) | Comma-separated list of nested arrays to fan out into their own topics (for example orders.line_items,products.variants,customers.addresses). See Fan-out. |
| Enable Dead Letter Queue (advanced) | true | Failed records are written to a DLQ topic instead of crashing the connector. |
5. Save and Copy the Endpoint
Save the source. Copy the Webhook URL and API Key — you will need them in the next section to configure Shopify.
Shopify Setup
Shopify does not allow custom headers on webhook deliveries — it only sends its own X-Shopify-* headers. Since Streamkap authenticates requests with an api_key, you must pass the key in the webhook URL as a query parameter:
https://<webhook-url>?api_key=<API_KEY>
Shopify will POST to this exact URL including the query string, and the connector authenticates the request via the api_key parameter.
1. Create an App in the Partners Dashboard
Note: As of January 2026, Shopify has deprecated legacy custom apps. New apps must be created via the Partners Dashboard. Existing legacy custom apps with permanent tokens still work but cannot be created on new stores.
- Go to the Shopify Partners Dashboard (create a partner account if needed).
- Click Apps → Create app → Create app manually.
- Name it (for example,
Streamkap), set the App URL to your Streamkap webhook URL.
- Click Create app.
Inside the app, go to Configuration → Access scopes and enable the scopes for the topics you plan to receive:
| Webhook Topics | Required Scope |
|---|
orders/create, orders/updated, orders/delete, orders/cancelled, orders/fulfilled, orders/paid | read_orders |
products/create, products/update, products/delete | read_products |
customers/create, customers/update, customers/delete | read_customers |
draft_orders/* | read_draft_orders |
fulfillments/* | read_orders |
inventory_items/*, inventory_levels/* | read_inventory |
collections/* | read_products |
refunds/create | read_orders |
Save. For a full CDC-style setup, enable read_orders, read_products, read_customers, and read_inventory at minimum.
3. Install the App and Get Credentials
- In the app, open Settings and note the Client ID and Client Secret.
- Install the app on your store (from the Partners Dashboard or via Settings → Apps and sales channels → Develop apps in your store admin) and approve the scopes.
You have two options for obtaining the access token Streamkap uses for snapshots.
Option A — Client credentials grant (recommended)
Paste the Client ID and Client Secret into the Streamkap source’s Auth tab. Streamkap handles token acquisition and refresh automatically (Shopify tokens are valid for 24 hours).
For webhook registration in the next step you still need a token. Mint one with curl:
curl -X POST "https://YOUR-STORE.myshopify.com/admin/oauth/access_token" \
-H "Content-Type: application/json" \
-d '{
"client_id": "YOUR_CLIENT_ID",
"client_secret": "YOUR_CLIENT_SECRET",
"grant_type": "client_credentials"
}'
The response contains an access_token valid for 24 hours.
Option B — Static access token (legacy custom apps only)
If you have an existing legacy custom app with a permanent token, paste it into the Streamkap source’s Auth tab as Access Token (Legacy). Leave Client ID / Client Secret empty.
4. Register Webhooks via the GraphQL Admin API
The Shopify Admin UI only registers one webhook at a time. For multiple topics, use the GraphQL Admin API.
Endpoint: https://YOUR-STORE.myshopify.com/admin/api/2024-10/graphql.json
Required header: X-Shopify-Access-Token: YOUR_ACCESS_TOKEN
Register a single webhook
curl -X POST "https://YOUR-STORE.myshopify.com/admin/api/2024-10/graphql.json" \
-H "Content-Type: application/json" \
-H "X-Shopify-Access-Token: YOUR_ACCESS_TOKEN" \
-d '{
"query": "mutation { webhookSubscriptionCreate(topic: ORDERS_CREATE, webhookSubscription: { callbackUrl: \"https://YOUR-WEBHOOK-URL?api_key=YOUR_API_KEY\", format: JSON }) { webhookSubscription { id } userErrors { field message } } }"
}'
Register all common webhooks at once
Save the following as setup-shopify-webhooks.sh, set the variables, and run it:
#!/bin/bash
STORE="YOUR-STORE.myshopify.com"
TOKEN="YOUR_ACCESS_TOKEN"
CALLBACK="https://YOUR-WEBHOOK-URL?api_key=YOUR_API_KEY"
API_VERSION="2024-10"
TOPICS=(
# Orders
ORDERS_CREATE ORDERS_UPDATED ORDERS_DELETE
ORDERS_CANCELLED ORDERS_FULFILLED ORDERS_PAID
# Products
PRODUCTS_CREATE PRODUCTS_UPDATE PRODUCTS_DELETE
# Customers
CUSTOMERS_CREATE CUSTOMERS_UPDATE CUSTOMERS_DELETE
# Draft Orders
DRAFT_ORDERS_CREATE DRAFT_ORDERS_UPDATE DRAFT_ORDERS_DELETE
# Fulfillments
FULFILLMENTS_CREATE FULFILLMENTS_UPDATE
# Inventory
INVENTORY_ITEMS_CREATE INVENTORY_ITEMS_UPDATE INVENTORY_ITEMS_DELETE
# Collections
COLLECTIONS_CREATE COLLECTIONS_UPDATE COLLECTIONS_DELETE
)
for TOPIC in "${TOPICS[@]}"; do
echo "Creating webhook for $TOPIC..."
curl -s -X POST "https://$STORE/admin/api/$API_VERSION/graphql.json" \
-H "Content-Type: application/json" \
-H "X-Shopify-Access-Token: $TOKEN" \
-d "{\"query\": \"mutation { webhookSubscriptionCreate(topic: $TOPIC, webhookSubscription: { callbackUrl: \\\"$CALLBACK\\\", format: JSON }) { webhookSubscription { id } userErrors { field message } } }\"}"
echo
done
Verify registered webhooks
curl -s -X POST "https://YOUR-STORE.myshopify.com/admin/api/2024-10/graphql.json" \
-H "Content-Type: application/json" \
-H "X-Shopify-Access-Token: YOUR_ACCESS_TOKEN" \
-d '{
"query": "{ webhookSubscriptions(first: 50) { edges { node { id topic endpoint { ... on WebhookHttpEndpoint { callbackUrl } } } } } }"
}'
Delete a webhook
curl -s -X POST "https://YOUR-STORE.myshopify.com/admin/api/2024-10/graphql.json" \
-H "Content-Type: application/json" \
-H "X-Shopify-Access-Token: YOUR_ACCESS_TOKEN" \
-d '{
"query": "mutation { webhookSubscriptionDelete(id: \"gid://shopify/WebhookSubscription/1234567890\") { deletedWebhookSubscriptionId userErrors { field message } } }"
}'
Via Shopify Admin UI (Quick Setup)
For a quick setup with a handful of topics:
- In Shopify Admin, go to Settings → Notifications → Webhooks.
- Click Create webhook.
- Select the event (for example
Order creation).
- Set format to JSON.
- Enter your Streamkap webhook URL with the API key appended:
https://YOUR-WEBHOOK-URL?api_key=YOUR_API_KEY.
- Save and repeat for each event.
Event Routing Reference
The payload router parses the X-Shopify-Topic header on each incoming request and routes by resource segment.
| Shopify Topic | Kafka Topic | Default Key |
|---|
orders/create, orders/updated, orders/delete, orders/cancelled, orders/fulfilled, orders/paid | orders | { id } |
products/create, products/update, products/delete | products | { id } |
customers/create, customers/update, customers/delete | customers | { id } |
draft_orders/* | draft_orders | { id } |
fulfillments/* | fulfillments | { id } |
inventory_items/* | inventory_items | { id } |
collections/* | collections | { id } |
shop/update | shop | (no key) |
Topics ending in /delete set __deleted: true on the output record so downstream sinks can issue tombstones. __op is emitted as a Kafka header (c create, u update, d delete, r snapshot).
Each record additionally carries __changeType (CREATE, UPDATE, DELETE, CANCELLED, FULFILLED, PAID, SNAPSHOT, …) in the value.
Fan-out
When Flatten Record Fields is on, scalar fields are promoted to the top level — but arrays of objects (line items, variants, addresses, tax lines, …) cannot be flattened cleanly. Fan-out emits one record per array element to a dedicated topic.
Set Fan-out Fields to a comma-separated list of resource.field pairs from the allowed set:
orders.line_items, orders.shipping_lines, orders.discount_codes, orders.tax_lines,
orders.fulfillments, products.variants, products.images, products.options,
customers.addresses, draft_orders.line_items, fulfillments.line_items
| Fan-out entry | Generated topic | Key shape |
|---|
orders.line_items | orders_line_items | { id: order_id, item_id: line_item_id } |
orders.shipping_lines | orders_shipping_lines | { id: order_id, item_id: shipping_line_id } |
orders.discount_codes | orders_discount_codes | { id: order_id } |
orders.tax_lines | orders_tax_lines | { id: order_id } |
orders.fulfillments | orders_fulfillments | { id: order_id, item_id: fulfillment_id } |
products.variants | products_variants | { id: product_id, item_id: variant_id } |
products.images | products_images | { id: product_id, item_id: image_id } |
products.options | products_options | { id: product_id, item_id: option_id } |
customers.addresses | customers_addresses | { id: customer_id, item_id: address_id } |
draft_orders.line_items | draft_orders_line_items | { id: draft_order_id, item_id: line_item_id } |
fulfillments.line_items | fulfillments_line_items | { id: fulfillment_id, item_id: line_item_id } |
Fan-out topics are added to the Schema list automatically — you do not need to register them manually. Streamkap rejects unknown fan-out entries when you save the source.
Each fan-out record contains the array element’s fields plus:
_ctx_event_id — the originating X-Shopify-Event-Id for correlation.
_ctx_shop_domain — the originating X-Shopify-Shop-Domain.
Fan-out topics do not emit tombstone records when array items are removed (for example when a line item is removed from an order). To handle deletions, configure the downstream sink for delete-and-reinsert on each parent event, or treat each parent record as the source of truth and full-replace the child rows.
HMAC Verification
To cryptographically verify that webhooks come from Shopify, set HMAC Secret in the Auth tab to your Shopify app’s client secret. Every payload is then verified against the X-Shopify-Hmac-Sha256 header using HMAC-SHA256. Failed payloads are rejected (and sent to the DLQ when enabled).
Leave the field empty to skip verification — useful while testing.
Snapshot
Once your source is live, you can run a snapshot from the Streamkap UI to load historical data for selected resources. Webhook streaming continues to run in parallel — snapshot is a one-time backfill, not an alternative to live events.
Snapshottable Resources
Only these resources can be snapshotted via Shopify’s GraphQL Admin API:
| Resource | What’s included |
|---|
orders | id, name, email, created/updated timestamps, total price, financial status, fulfillment status, customer, line items |
products | id, title, handle, status, vendor, type, created/updated timestamps, variants, images |
customers | id, name, email, phone, created/updated timestamps, state, number of orders, addresses |
draft_orders | id, name, status, created/updated timestamps, line items |
collections | id, title, handle, updated timestamp, sort order |
inventory_items | id, sku, created/updated timestamps, requires shipping, tracked |
All other resources you select in the Schema tab (fulfillments, shop, discounts, carts, etc.) cannot be snapshotted — they only receive live events via webhooks once you register them in Shopify. This is a limitation of the Shopify Admin API, which exposes historical-fetch endpoints only for the resources above.
Troubleshooting
| Issue | Check |
|---|
401 Unauthorized on webhook delivery | Verify the URL registered in Shopify includes ?api_key=<API_KEY> with the value from the Streamkap source. Shopify does not pass custom headers, so the key must be in the query string. |
| Webhook not firing | In Shopify Admin Settings → Notifications → Webhooks, check delivery status. Shopify retries 8 times over 4 hours and then deletes the subscription. |
Callback URL is not allowed | The callback URL must be HTTPS. For local testing, expose Streamkap behind a tunnel (ngrok, Cloudflare Tunnel). |
| Snapshot does not start | Confirm Store URL is populated and either Client ID + Client Secret or Access Token is set in the Auth tab. |
Access denied on snapshot | Verify the app has the right scopes for the resources you are snapshotting. Reinstall the app after changing scopes. |
Events arrive in the unknown topic | The resource segment in X-Shopify-Topic is not in your Schema list. Add it, or set Unselected Resource Behavior to SKIP. |
400 Bad Request saving the source | The source rejects unknown fan-out fields. The error message lists the allowed values. |
| HMAC failures filling the DLQ | The HMAC secret must be the client secret of the Shopify app that registered the webhooks. Confirm you copied it from the same app. |
Limitations
- The Shopify Webhook source is currently Beta.
- Shopify expects a
200 response within 5 seconds. The connector responds immediately and processes asynchronously; failures are captured in the DLQ when enabled.
- Snapshot supports only the resources backed by the GraphQL Admin API listed above. Other resources (e.g.,
fulfillments, shop) are streamed via webhooks only.
- Mandatory Shopify compliance webhooks (
customers/data_request, customers/redact, shop/redact) are accepted by the endpoint but currently routed to the unknown topic; you should respond to them out of band per Shopify’s requirements.
- Maximum payload size is 50 MB and maximum header size is 64 KB.
- The connector runs as a single task; horizontal scaling requires multiple source instances.
See Also