Get dashboard lineage topics

Returns detailed topic-level lineage information including source, transforms, destinations, and per-topic metrics. Supports filtering and pagination.

GET/dashboard/lineage/topics

Authorization

AuthorizationBearer token · headerrequired

Query parameters

source_idstring

Filter by source entity ID

destination_idstring

Filter by destination entity ID

transform_idstring

Filter by transform entity ID

topic_namestring

Filter by topic name (substring match)

filterstring

Comma-separated topic filters: has_destination, orphaned, has_transform, dlq, snapshotting

include_dlqboolean

When false, exclude DLQ topics

default: true

snapshotstring

Comma-separated snapshot status filter matched against each table's own status: running, pending, completed, completed_with_warnings, failed, cancelled, paused.

pageinteger

Page number (1-indexed)

min 1 · default: 1

page_sizeinteger

Results per page

min 1 · max 100 · default: 5

sortstring

Sort field

default: "byteTotal"

Allowed:byteTotallatencyspeedlagname

sort_dirstring

Sort direction

default: "desc"

Allowed:ascdesc

include_metricsboolean

When false, skip ClickHouse metrics and return metrics as null for faster initial render

default: true

Responses

200

Successful Response

pageinteger

Current page number

default: 1

page_sizeinteger

Results per page

default: 10

totalinteger | null

Total number of results

Show properties

Any of:

integer

integer

null

null

has_nextboolean | null

Whether more pages exist

Show properties

Any of:

boolean

boolean

null

null

resultLineageTopicRes[]

List of topics with lineage info

Show properties

Array of LineageTopicRes

idstringrequired

Topic entity ID

namestringrequired

Full topic name

sourceTopicSourceRefrequired

Source that produces this topic

Show properties

idstringrequired

Source entity ID

namestringrequired

Source display name

connectorstring

Source connector type

default: ""

snapshot_progressSourceSnapshotProgressRes | null

Snapshot progress. Null when include_metrics=false or no snapshot data.

Show properties

Any of:

SourceSnapshotProgressRes

statestringrequired

Snapshot state: running, pending, completed, failed, or not_started

snapshot_typestring

Snapshot type: incremental or blocking

default: "incremental"

total_topic_countinteger

Logical topics (selectors) in the snapshot — what the user sees. Equals total_table_count unless a selector fanned out (sharded/regex).

default: 0

remaining_topic_countinteger

Logical topics still to be processed

default: 0

done_topic_countinteger

Logical topics in a terminal DONE/COMPLETED state

default: 0

cancelled_topic_countinteger

Logical topics cancelled by the user before completing

default: 0

failed_topic_countinteger

Logical topics that terminated in a FAILED state

default: 0

total_table_countinteger

Physical tables in the snapshot — expanded per shard/regex match. Exceeds total_topic_count only on fanout.

default: 0

remaining_table_countinteger

Physical tables still to be processed

default: 0

rows_totalinteger

Total source rows across all tables (from QA metrics). 0 if unavailable.

default: 0

rows_scannedinteger

Total rows scanned across all tables. 0 if unavailable.

default: 0

progress_typestring

Whether percent_complete is based on row counts ('rows') or table counts ('tables').

default: "tables"

Allowed:rowstables

percent_completenumber

Snapshot completion percentage (0-100)

default: 0

eta_msinteger

Estimated milliseconds to completion

default: 0

start_timestampstring | null

Snapshot start time (UTC)

Show properties

Any of:

string

string

null

null

end_timestampstring | null

Snapshot end time (UTC)

Show properties

Any of:

string

string

null

null

null

null

metricsTopicMetricsDetailRes | null

Topic-level metrics. Null when include_metrics=false.

Show properties

Any of:

TopicMetricsDetailRes

StreamingMilliSecondsBehindSourceinteger

Streaming latency in milliseconds

default: 0

SnapshotPercentageCompleteinteger

Snapshot completion percentage (uppercase)

default: 0

recordSendTotalinteger

Total records sent

default: 0

sourceRecordWriteTotalinteger

Total source records written

default: 0

snapshotPercentageCompleteinteger

Snapshot completion percentage (lowercase)

default: 0

recordsLaginteger

Consumer records lag

default: 0

byteTotalinteger

Total bytes processed

default: 0

sourceRecordPollRatenumber

Source record poll rate (records/sec)

default: 0

sourceQaCountnumber | null

Source DB row count from metrics_qa. Null when not collected.

Show properties

Any of:

number

number

null

null

null

null

snapshot_statusstring | null

Latest snapshot status for this topic: running, pending, completed, failed, or null if never snapshotted.

Show properties

Any of:

string

string

null

null

snapshot_detailsTopicSnapshotStatusRes[]

Detailed snapshot history for this topic. Empty list if never snapshotted.

Show properties

Array of TopicSnapshotStatusRes

statusstringrequired

Snapshot status: running, pending, completed, failed

start_timestampstring | null

Snapshot start time (UTC)

Show properties

Any of:

string

string

null

null

end_timestampstring | null

Snapshot end time (UTC)

Show properties

Any of:

string

string

null

null

duration_msinteger | null

Snapshot duration in milliseconds

Show properties

Any of:

integer

integer

null

null

rows_scannedinteger

Number of rows scanned so far

default: 0

rows_totalinteger

Total number of rows to scan

default: 0

tracestring | null

Error trace message for failed snapshots

Show properties

Any of:

string

string

null

null

eta_msinteger

Estimated milliseconds to completion

default: 0

transformsTopicTransformRef[]

Transforms applied to this topic

Show properties

Array of TopicTransformRef

idstringrequired

Transform entity ID

namestringrequired

Transform display name

statusstring

Transform status (e.g. RUNNING)

default: ""

metricsTransformMetricsRes | null

Transform-level metrics. Null when include_metrics=false.

Show properties

Any of:

TransformMetricsRes

latencyinteger

Transform processing latency in milliseconds

default: 0

null

null

output_topicsstring[]

Output topics from this transform

transform_typeTransformsLibraryEnum | null

Transform library type (e.g. topic_router, sql_join)

Show properties

Any of:

TransformsLibraryEnum

string

null

null

destinationsTopicDestinationRef[]

Destinations consuming this transform's output topics, with lag/count keyed to the output topic.

destinationsTopicDestinationRef[]

Destinations consuming this topic

Show properties

Array of TopicDestinationRef

idstringrequired

Destination entity ID

namestringrequired

Destination display name

connectorstring

Destination connector type

default: ""

pipeline_idstring

Pipeline entity ID

default: ""

pipeline_statusstring

Pipeline status

default: ""

laginteger | null

Destination consumer lag. Null when include_metrics=false.

Show properties

Any of:

integer

integer

null

null

latency_msinteger | null

Destination wall-clock consumer latency in milliseconds (worst partition). Null when include_metrics=false or no probe sample exists yet.

Show properties

Any of:

integer

integer

null

null

qaCountnumber | null

Destination row count for this topic from metrics_qa. Null when not collected.

Show properties

Any of:

number

number

null

null

e2e_eta_msinteger | null

Estimated milliseconds for destination to fully consume remaining data. Shown during active snapshots or when lag exceeds threshold. -1 means records are remaining but destination is not consuming (stalled). Null when not applicable.

Show properties

Any of:

integer

integer

null

null

422

Validation Error

detailValidationError[]

Show properties

Array of ValidationError

locstring | integer[]required

Show properties

Array of string | integer

Any of:

string

string

integer

integer

msgstringrequired

typestringrequired

inputany

ctxobject

Request

curl -X GET 'https://api.streamkap.com/dashboard/lineage/topics' \
  -H 'Authorization: Bearer YOUR_TOKEN' \
  -H 'Accept: application/json'

const response = await fetch("https://api.streamkap.com/dashboard/lineage/topics", {
  method: "GET",
  headers: {
    "Authorization": "Bearer YOUR_TOKEN",
    "Accept": "application/json"
  }
});

import requests

response = requests.get(
    "https://api.streamkap.com/dashboard/lineage/topics",
    headers={
        "Authorization": "Bearer YOUR_TOKEN",
        "Accept": "application/json"
    },
)

Response

{
  "page": 1,
  "page_size": 10,
  "total": 0,
  "has_next": true,
  "result": [
    {
      "id": "string",
      "name": "string",
      "source": {
        "id": "string",
        "name": "string",
        "connector": "",
        "snapshot_progress": {
          "state": "string",
          "snapshot_type": "incremental",
          "total_topic_count": 0,
          "remaining_topic_count": 0,
          "done_topic_count": 0,
          "cancelled_topic_count": 0,
          "failed_topic_count": 0,
          "total_table_count": 0,
          "remaining_table_count": 0,
          "rows_total": 0,
          "rows_scanned": 0,
          "progress_type": "rows",
          "percent_complete": 0,
          "eta_ms": 0,
          "start_timestamp": "string",
          "end_timestamp": "string"
        }
      },
      "metrics": {
        "StreamingMilliSecondsBehindSource": 0,
        "SnapshotPercentageComplete": 0,
        "recordSendTotal": 0,
        "sourceRecordWriteTotal": 0,
        "snapshotPercentageComplete": 0,
        "recordsLag": 0,
        "byteTotal": 0,
        "sourceRecordPollRate": 0,
        "sourceQaCount": 0
      },
      "snapshot_status": "string",
      "snapshot_details": [
        {
          "status": "string",
          "start_timestamp": "string",
          "end_timestamp": "string",
          "duration_ms": 0,
          "rows_scanned": 0,
          "rows_total": 0,
          "trace": "string",
          "eta_ms": 0
        }
      ],
      "transforms": [
        {
          "id": "string",
          "name": "string",
          "status": "",
          "metrics": {
            "latency": 0
          },
          "output_topics": [
            "string"
          ],
          "transform_type": "fan_out",
          "destinations": [
            {
              "id": "string",
              "name": "string",
              "connector": "",
              "pipeline_id": "",
              "pipeline_status": "",
              "lag": 0,
              "latency_ms": 0,
              "qaCount": 0,
              "e2e_eta_ms": 0
            }
          ]
        }
      ],
      "destinations": [
        {
          "id": "string",
          "name": "string",
          "connector": "",
          "pipeline_id": "",
          "pipeline_status": "",
          "lag": 0,
          "latency_ms": 0,
          "qaCount": 0,
          "e2e_eta_ms": 0
        }
      ]
    }
  ]
}

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string",
      "input": null,
      "ctx": {}
    }
  ]
}