Get dashboard lineage topics

Returns detailed topic-level lineage information including source, transforms, destinations, and per-topic metrics. Supports filtering and pagination.

GET/dashboard/lineage/topics

Authorization

AuthorizationBearer token · headerrequired

Query parameters

source_idstring

Filter by source entity ID

destination_idstring

Filter by destination entity ID

transform_idstring

Filter by transform entity ID

topic_namestring

Filter by topic name (substring match)

filterstring

Comma-separated topic filters: has_destination, orphaned, has_transform, dlq, snapshotting

include_dlqboolean

When false, exclude DLQ topics

default: true

snapshotstring

Comma-separated snapshot status filter: running, completed, pending

pageinteger

Page number (1-indexed)

min 1 · default: 1

page_sizeinteger

Results per page

min 1 · max 100 · default: 5

sortstring

Sort field

default: "byteTotal"

Allowed:byteTotallatencyspeedlagname

sort_dirstring

Sort direction

default: "desc"

Allowed:ascdesc

include_metricsboolean

When false, skip ClickHouse metrics and return metrics as null for faster initial render

default: true

Responses

200

Successful Response

pageinteger

Current page number

default: 1

page_sizeinteger

Results per page

default: 10

totalinteger | null

Total number of results

Show properties

Any of:

integer

integer

null

null

has_nextboolean | null

Whether more pages exist

Show properties

Any of:

boolean

boolean

null

null

resultLineageTopicRes[]

List of topics with lineage info

Show properties

Array of LineageTopicRes

idstringrequired

Topic entity ID

namestringrequired

Full topic name

sourceTopicSourceRefrequired

Reference to the source that produces a topic.

Show properties

idstringrequired

Source entity ID

namestringrequired

Source display name

connectorstring

Source connector type

default: ""

snapshot_progressSourceSnapshotProgressRes | null

Snapshot progress. Null when include_metrics=false or no snapshot data.

Show properties

Any of:

SourceSnapshotProgressRes

statestringrequired

Snapshot state: running, pending, completed, failed, or not_started

snapshot_typestring

Snapshot type: incremental or blocking

default: "incremental"

total_topic_countinteger

Logical topics (selectors) in the snapshot — what the user sees. Equals total_table_count unless a selector fanned out (sharded/regex).

default: 0

remaining_topic_countinteger

Logical topics still to be processed

default: 0

done_topic_countinteger

Logical topics in a terminal DONE/COMPLETED state

default: 0

cancelled_topic_countinteger

Logical topics cancelled by the user before completing

default: 0

failed_topic_countinteger

Logical topics that terminated in a FAILED state

default: 0

total_table_countinteger

Physical tables in the snapshot — expanded per shard/regex match. Exceeds total_topic_count only on fanout.

default: 0

remaining_table_countinteger

Physical tables still to be processed

default: 0

rows_totalinteger

Total source rows across all tables (from QA metrics). 0 if unavailable.

default: 0

rows_scannedinteger

Total rows scanned across all tables. 0 if unavailable.

default: 0

progress_typestring

Whether percent_complete is based on row counts ('rows') or table counts ('tables').

default: "tables"

Allowed:rowstables

percent_completenumber

Snapshot completion percentage (0-100)

default: 0

eta_msinteger

Estimated milliseconds to completion

default: 0

start_timestampstring | null

Snapshot start time (UTC)

Show properties

Any of:

string

string

null

null

end_timestampstring | null

Snapshot end time (UTC)

Show properties

Any of:

string

string

null

null

null

null

metricsTopicMetricsDetailRes | null

Topic-level metrics. Null when include_metrics=false.

Show properties

Any of:

TopicMetricsDetailRes

StreamingMilliSecondsBehindSourceinteger

Streaming latency in milliseconds

default: 0

SnapshotPercentageCompleteinteger

Snapshot completion percentage (uppercase)

default: 0

recordSendTotalinteger

Total records sent

default: 0

sourceRecordWriteTotalinteger

Total source records written

default: 0

snapshotPercentageCompleteinteger

Snapshot completion percentage (lowercase)

default: 0

recordsLaginteger

Consumer records lag

default: 0

byteTotalinteger

Total bytes processed

default: 0

sourceRecordPollRatenumber

Source record poll rate (records/sec)

default: 0

sourceQaCountnumber | null

Source DB row count from metrics_qa. Null when not collected.

Show properties

Any of:

number

number

null

null

null

null

snapshot_statusstring | null

Latest snapshot status for this topic: running, pending, completed, failed, or null if never snapshotted.

Show properties

Any of:

string

string

null

null

snapshot_detailsTopicSnapshotStatusRes[]

Detailed snapshot history for this topic. Empty list if never snapshotted.

Show properties

Array of TopicSnapshotStatusRes

statusstringrequired

Snapshot status: running, pending, completed, failed

start_timestampstring | null

Snapshot start time (UTC)

Show properties

Any of:

string

string

null

null

end_timestampstring | null

Snapshot end time (UTC)

Show properties

Any of:

string

string

null

null

duration_msinteger | null

Snapshot duration in milliseconds

Show properties

Any of:

integer

integer

null

null

rows_scannedinteger

Number of rows scanned so far

default: 0

rows_totalinteger

Total number of rows to scan

default: 0

tracestring | null

Error trace message for failed snapshots

Show properties

Any of:

string

string

null

null

eta_msinteger

Estimated milliseconds to completion

default: 0

transformsTopicTransformRef[]

Transforms applied to this topic

Show properties

Array of TopicTransformRef

idstringrequired

Transform entity ID

namestringrequired

Transform display name

statusstring

Transform status (e.g. RUNNING)

default: ""

metricsTransformMetricsRes | null

Transform-level metrics. Null when include_metrics=false.

Show properties

Any of:

TransformMetricsRes

latencyinteger

Transform processing latency in milliseconds

default: 0

null

null

output_topicsstring[]

Output topics from this transform

transform_typeTransformsLibraryEnum | null

Transform library type (e.g. topic_router, sql_join)

Show properties

Any of:

TransformsLibraryEnum

string

null

null

destinationsTopicDestinationRef[]

Destinations consuming this transform's output topics, with lag/count keyed to the output topic.

destinationsTopicDestinationRef[]

Destinations consuming this topic

Show properties

Array of TopicDestinationRef

idstringrequired

Destination entity ID

namestringrequired

Destination display name

connectorstring

Destination connector type

default: ""

pipeline_idstring

Pipeline entity ID

default: ""

pipeline_statusstring

Pipeline status

default: ""

laginteger | null

Destination consumer lag. Null when include_metrics=false.

Show properties

Any of:

integer

integer

null

null

latency_msinteger | null

Destination wall-clock consumer latency in milliseconds (worst partition). Null when include_metrics=false or no probe sample exists yet.

Show properties

Any of:

integer

integer

null

null

qaCountnumber | null

Destination row count for this topic from metrics_qa. Null when not collected.

Show properties

Any of:

number

number

null

null

e2e_eta_msinteger | null

Estimated milliseconds for destination to fully consume remaining data. Shown during active snapshots or when lag exceeds threshold. -1 means records are remaining but destination is not consuming (stalled). Null when not applicable.

Show properties

Any of:

integer

integer

null

null

422

Validation Error

detailValidationError[]

Show properties

Array of ValidationError

locstring | integer[]required

Show properties

Array of string | integer

Any of:

string

string

integer

integer

msgstringrequired

typestringrequired

inputany

ctxobject

Request

curl -X GET 'https://api.streamkap.com/dashboard/lineage/topics' \
  -H 'Authorization: Bearer YOUR_TOKEN'

const response = await fetch("https://api.streamkap.com/dashboard/lineage/topics", {
  method: "GET",
  headers: {
    "Authorization": "Bearer YOUR_TOKEN"
  }
});

import requests

response = requests.get(
    "https://api.streamkap.com/dashboard/lineage/topics",
    headers={
        "Authorization": "Bearer YOUR_TOKEN"
    },
)

Response

{
  "page": 1,
  "page_size": 10,
  "total": 0,
  "has_next": true,
  "result": [
    {
      "id": "string",
      "name": "string",
      "source": {
        "id": "string",
        "name": "string",
        "connector": "",
        "snapshot_progress": {
          "state": "string",
          "snapshot_type": "incremental",
          "total_topic_count": 0,
          "remaining_topic_count": 0,
          "done_topic_count": 0,
          "cancelled_topic_count": 0,
          "failed_topic_count": 0,
          "total_table_count": 0,
          "remaining_table_count": 0,
          "rows_total": 0,
          "rows_scanned": 0,
          "progress_type": "rows",
          "percent_complete": 0,
          "eta_ms": 0,
          "start_timestamp": "string",
          "end_timestamp": "string"
        }
      },
      "metrics": {
        "StreamingMilliSecondsBehindSource": 0,
        "SnapshotPercentageComplete": 0,
        "recordSendTotal": 0,
        "sourceRecordWriteTotal": 0,
        "snapshotPercentageComplete": 0,
        "recordsLag": 0,
        "byteTotal": 0,
        "sourceRecordPollRate": 0,
        "sourceQaCount": 0
      },
      "snapshot_status": "string",
      "snapshot_details": [
        {
          "status": "string",
          "start_timestamp": "string",
          "end_timestamp": "string",
          "duration_ms": 0,
          "rows_scanned": 0,
          "rows_total": 0,
          "trace": "string",
          "eta_ms": 0
        }
      ],
      "transforms": [
        {
          "id": "string",
          "name": "string",
          "status": "",
          "metrics": {
            "latency": 0
          },
          "output_topics": [
            "string"
          ],
          "transform_type": "fan_out",
          "destinations": [
            {
              "id": "string",
              "name": "string",
              "connector": "",
              "pipeline_id": "",
              "pipeline_status": "",
              "lag": 0,
              "latency_ms": 0,
              "qaCount": 0,
              "e2e_eta_ms": 0
            }
          ]
        }
      ],
      "destinations": [
        {
          "id": "string",
          "name": "string",
          "connector": "",
          "pipeline_id": "",
          "pipeline_status": "",
          "lag": 0,
          "latency_ms": 0,
          "qaCount": 0,
          "e2e_eta_ms": 0
        }
      ]
    }
  ]
}

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string",
      "input": null,
      "ctx": {}
    }
  ]
}