Streamkap can perform snapshots to backfill your data.
- Please ensure you've followed your connector setup guide to enable this process.
- All snapshots/backfills are free of charge.
- The default behavior is to carry out an initial snapshot across all tables when you create the connector
- After this completes, the connector will switch to reading the log data and streaming mode
- If you restart or make configuration changes during the above snapshot process, the snapshot will stop and will not resume. You should trigger an adhoc snapshot.
- Additional tables added after the connector is created will not be snapshotted by default. You should trigger an adhoc snapshot.
After the connector is created, you can perform adhoc incremental snapshots, which will:
- Sync historical data from today's date and working its way backwards using a process called watermarking.
- It will do this in 1024 row limit increments unless you increase this value in the connector settings.
It's possible to trigger a snapshot via the connector or at table level.
This will trigger a snapshot across all tables/topics.
This will carry out a snapshot only for the selected table/topic.
Upon triggering a snapshot, the status will update
It is possible to cancel the snapshot while it's in pending status
Each connector has a setting called Snapshot Chunk Size.
This represents the number of rows that are fetched upon request which will default at 1024 rows. Increasing this value will linearly increase the speed of the snapshots.
Updated 6 months ago