API Sources

API sources poll a SaaS vendor's REST API on a schedule, write each resource you select to its own topic, and deliver with Send to destination.

An API source reads records from a SaaS application’s REST API on a schedule and writes them to Kafka topics, one topic per resource you select, named source_<id>.<vendor>.<resource>. HubSpot, Salesforce, NetSuite, Stripe, Google Analytics 4, Facebook Ads, Google Ads and Zendesk are API sources.

API sources list on the Sources page under the API tab. They are enabled per organization: if the one you need is not in Add Connectors, contact Streamkap.

How an API Source Differs from a CDC Source

A CDC source reads a database’s change log. A vendor’s REST API has no change log, so an API source polls: it asks for records changed since the last poll, and repeats.

Records land flat in your destination like any other Streamkap source, and every existing destination consumes them unchanged. Each resource’s first poll reads it in full, from the backfill start date; later polls read only what changed. A poll where nothing changed writes zero records.

Create an API Source

Test Connection and Test Access

Both buttons run the same read-only test, and nothing is created or written.

• Test connection, beside the credential, says whether the credential works and which vendor account it belongs to. It also warns when that account already backs another source in this workspace. A credential that works but needs attention, such as a token close to expiry, passes with a Check this connection warning.
• Test access, under Resources, reads one row of each selected resource with the same request a poll sends. The result opens with a count such as “24 readable · 5 blocked · 2 not checked”.
  • Blocked resources are grouped by what fixes them, one line per missing permission naming its resources. Each group has a Deselect N button (Deselect and the resource’s name for one), so you can drop them or grant the access and test again.
  • Not checked resources are ones the test stopped before reading, because it paces its requests to the vendor. Test again with fewer selected.
  • A resource that is readable but may return nothing as configured passes with a warning beside its name.

On the create form, while a selected resource is known to be blocked, the result says “Create is held until these resources are readable or deselected” and Create stays disabled. Saving an edit is never held. Every create and update is checked again on save.

Backfill Start Date

Backfill start date bounds how far back each resource’s first sync reads. Keep All historical data (or the vendor’s default horizon, where the connector page names one), or choose Specific start date. It also applies to resources added later; changing it does not re-read resources already synced.

Send Topics to a Destination

API sources do not appear in the Create pipeline source picker. You send their topics to a destination instead, from whichever end you are on:

• From the source: open its Destinations tab and click Send topics. Choose all topics or pick some, then a destination.
• From the Topics page: select topics with their checkboxes and click Send to destination in the bulk actions bar. CDC topics cannot be sent this way; build a pipeline for those.
• From the destination: click Add topics next to the Topics heading.

Before you confirm, the picker shows how many topics go to which destination and which already deliver there. Streamkap creates and maintains the pipeline. It appears on the Pipelines page with an Auto-managed chip that links to the source’s Destinations tab, the one place it is changed or removed.

To stop delivering, go to the source’s Destinations tab: remove individual topics, or click Detach all to clear a destination. Removing the last topic removes the destination from the source.

Editing an API Source

Open the source and edit its Auth or Settings tab.

• Credentials: a stored secret stays as it is unless you click Replace and enter a new one. An optional secret left blank after Replace is removed on save.
• A new account location: to change the Zendesk subdomain, click Connect with Zendesk again before you save. To change the Salesforce My Domain URL or the NetSuite Account ID, enter the new account’s credentials in the same save.
• Resources: before you save, a note under the list says what the change does, and Test access checks any resource you added.
  • An added resource is backfilled from the Backfill start date and sent to every destination on the source’s Destinations tab.
  • A removed resource stops syncing and is detached from those destinations. Its destination table and rows are kept.
  • A destination left with nothing to receive is disconnected from the source; the note names it before you save. Adding a resource in the same save keeps it connected.
  • A resource added back is backfilled again from the start. Rows deleted in the vendor while it was removed stay in the destination.
  • A topic you removed from one destination on the Destinations tab stays removed when you edit the source; only added resources go to every destination.

If a destination cannot be updated, the source still saves and shows Saved with a warning, naming the destination and what to do.

Deletes

A delete arrives as a record with __deleted = true, and your destination marks the row deleted. How the connector learns of it depends on the vendor:

• From the vendor’s event log, with the other changes, in order. Stripe works this way.
• From a sweep of the vendor’s deleted or archived records, on a fixed interval.
• From a re-read: a list with no deleted view is read whole, and a row missing from a complete read is deleted. A read that stops short deletes nothing.
• From a reconciliation of live IDs, daily or weekly, for deletes the sweep cannot see.

A swept, re-read or reconciled delete arrives up to one interval after inserts and updates. Each connector page says which applies to which resource, and which cannot report deletes.

Delete coverage. A vendor keeps deleted records readable for a limited time: Salesforce 15 days, Stripe’s events and Zendesk’s deleted tickets and users 30, HubSpot’s archived records 90. When a resource has not read its deletes for two thirds of that window, because the source is failing or paused, its status says “Delete coverage at risk”, naming the resource and the UTC time to read them by, then “Delete coverage lapsing” once that time passes. Fix or resume the source before then. A delete that leaves the vendor unread is caught later only by a reconciliation, where the connector has one.

Some vendors report removals as updates, or not at all:

• Google Ads: removed campaigns, ad groups, ads and criteria arrive as upserts with status REMOVED; its daily re-reads still delete a row missing from a complete read.
• Facebook Ads: Meta does not serve deleted campaigns, ad sets or ads, so their deletion never reaches the destination.
• Google Analytics 4 and Facebook Ads insights: report rows for recent days are restated, not deleted.

Connector Status

• Broken: one resource’s polls keep failing, for example during an outage of one vendor endpoint. The source stays Broken with that resource’s error until it polls cleanly again, while the other resources keep streaming, so an alert sees one sustained Broken rather than one that clears at every poll.
• Throttled: the connector paces itself below the vendor’s rate limits, leaving headroom for your other integrations, and stops before it spends a reported quota. It then shows Throttled with the time it resumes, and resumes on its own without losing data.
• A resource you lack access to pauses on its own, where the vendor reports access per resource, and recovers once you grant access.
• A refused child: where one resource reads several children (Google Ads and Facebook Ads accounts, GA4 properties, Zendesk tickets’ audits and comments, the objects behind HubSpot properties), a child the vendor refuses is skipped with a warning and its rows already delivered stay. Only when every child is refused, the resource reads just one, or (for Zendesk’s per-ticket resources) 100 tickets in a row refuse, does the refusal stop it.

New fields are read from your account, not hardcoded, so a property you add appears without reconfiguring Streamkap; your destination adds the column if it evolves its schema. GA4, Facebook Ads and Google Ads are the exceptions: their columns are fixed by the report definition or API version. Zendesk custom field values arrive as JSON text in custom_fields and user_fields, not as new columns.

One Source per Vendor Account

Each vendor account can back only one source per workspace: one HubSpot portal, Salesforce org, NetSuite account, Stripe account and mode, Google Analytics 4 property, Facebook Ads ad account, Google Ads customer ID, or Zendesk account. A second source for the same account is rejected on save with “A source for this vendor account already exists in this workspace.”

Available API Sources

Using the API

API sources are created and updated with the same POST /sources and PUT /sources/{source_id} endpoints as any other source. To link one to a destination, use the topic-destination endpoints; POST /pipelines rejects an API source. See Send an API Source to a Destination.

Related Documentation

• Sources - manage sources, their status and config history
• Pipelines - connect CDC sources to destinations
• Topics - browse topics and inspect their data