Schema Evolution
Schema evolution enables a destination to adapt the schema of its objects (e.g. columns, fields, tables, files, collections) over time without disrupting operations or data integrity.
Streamkap Connectors support - where applicable - the following types of schema changes:
Object | Change | Behavior |
---|---|---|
Column | Add | Adds the new column to the end of the destination table. |
Remove | Keeps the column in the destination table intact but is left empty (NULL values) from the point of change onwards. | |
Data type | Adds a new column. For example if there is a column named AGE which is the NUMBER type but is then altered to VARCHAR type, an additional column is added named AGE_CHR of type VARCHAR . AGE column is kept intact. | |
Rename | Adds a new column. For example, if there is a column named FIRST_NAME that's renamed to FULL_NAME , an additional column is added named FULL_NAME and will be populated from the point of change onwards. FIRST_NAME column is kept intact but is left empty (NULL values). | |
Table | Add | Adding a table is done by modifying your Source Connector and Pipelines. Streamkap will automatically backfill/snapshot the newly added table (by default) while streaming data simultaneously. |
Remove | Keeps the destination table intact but stops capturing change event data for it. | |
Rename | Renaming table(s) will report an error and you will need to add the renamed table to your affected Source Connector and Pipeline(s) to capture it. |
Updated 9 days ago