Skip to main content
POST
/
transforms
/
full-process
Create New Transform With Full Process
curl --request POST \
  --url https://api.streamkap.com/transforms/full-process \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "transform": "fan_out",
  "config": {
    "transforms.name": "<string>",
    "transforms.input.topic.pattern": "<string>",
    "transforms.output.topic.pattern": "<string>",
    "transforms.input.serialization.format": "Any",
    "transforms.output.serialization.format": "Any",
    "transforms.language": "<string>",
    "transforms.input.job.parallelism": 5,
    "transforms.topic.ttl": "<string>"
  },
  "implementation": {
    "language": "PYTHON",
    "value_transform": "<string>",
    "key_transform": "<string>",
    "topic_transform": "<string>",
    "common_transform": "<string>",
    "value_schema_transform": "<string>",
    "key_schema_transform": "<string>",
    "unit_test": {
      "test_name": "<string>",
      "input__json": [
        {
          "topic": "<string>",
          "value": {},
          "key": {},
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "output__json": [
        {
          "topic": "<string>",
          "value": {},
          "key": {},
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "result__json": [
        {
          "topic": "<string>",
          "value": {},
          "key": {},
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "is_success": true,
      "html_diff": "<string>",
      "text_diff": "<string>"
    }
  },
  "created_from": "api"
}
'
{
  "id": "<string>",
  "name": "<string>",
  "sub_id": "<string>",
  "tenant_id": "<string>",
  "transform_type": "fan_out",
  "service_id": "<string>",
  "start_time": "<string>",
  "job_name": "<string>",
  "topic_ids": [
    "<string>"
  ],
  "config": {
    "transforms.name": "<string>",
    "transforms.input.topic.pattern": "<string>",
    "transforms.output.topic.pattern": "<string>",
    "transforms.input.serialization.format": "Any",
    "transforms.output.serialization.format": "Any",
    "transforms.language": "<string>",
    "transforms.input.job.parallelism": 5,
    "transforms.topic.ttl": "<string>"
  },
  "implementation": {
    "language": "PYTHON",
    "value_transform": "<string>",
    "key_transform": "<string>",
    "topic_transform": "<string>",
    "common_transform": "<string>",
    "value_schema_transform": "<string>",
    "key_schema_transform": "<string>",
    "unit_test": {
      "test_name": "<string>",
      "input__json": [
        {
          "topic": "<string>",
          "value": {},
          "key": {},
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "output__json": [
        {
          "topic": "<string>",
          "value": {},
          "key": {},
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "result__json": [
        {
          "topic": "<string>",
          "value": {},
          "key": {},
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "is_success": true,
      "html_diff": "<string>",
      "text_diff": "<string>"
    }
  },
  "metrics": {
    "tasks": 123,
    "latency": 123,
    "status": "<string>",
    "duration": 123,
    "start_time": 123,
    "end_time": "<string>",
    "checkpoints": {
      "job_id": "<string>",
      "counts": {},
      "summary": {},
      "latest": {
        "id": 123,
        "status": "<string>",
        "is_savepoint": true,
        "trigger_timestamp": 123,
        "latest_ack_timestamp": 123,
        "state_size": 123,
        "end_to_end_duration": 123,
        "alignment_buffered": 123,
        "num_subtasks": 123,
        "num_acknowledged_subtasks": 123
      },
      "history": [],
      "operator_breakdown": []
    }
  },
  "live_version": "<string>",
  "preview_version": "<string>",
  "version": 1,
  "updated_timestamp": "2023-11-07T05:31:56Z",
  "implementation_updated_timestamp": "2023-11-07T05:31:56Z",
  "description": "<string>",
  "history_created_timestamp": "2023-11-07T05:31:56Z",
  "history_updated_timestamp": "2023-11-07T05:31:56Z"
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Query Parameters

​
secret_returned
boolean
default:false

Whether to include secret values in the response

​
replay_window
string
default:""

Body

application/json
​
transform
enum<string>
required
Available options:
fan_out,
sql_join,
enrich,
enrich_async,
map_filter,
toast_handling,
un_nesting,
rollup
​
config
TransformConfig · object
required
​
implementation
TransformsMapFilterImplementation · object
required
​
created_from
enum<string> | null
default:api

Origin of an entity in the system.

Tracks how entities (sources, destinations, pipelines) were created, enabling audit trails and origin-specific behavior.

Available options:
terraform,
web,
api

Response

Successful Response

Basic transform details.

​
id
string
required
​
name
string
required
​
sub_id
string
required
​
tenant_id
string
required
​
transform_type
enum<string>
required
Available options:
fan_out,
sql_join,
enrich,
enrich_async,
map_filter,
toast_handling,
un_nesting,
rollup
​
service_id
string | null
​
start_time
​
job_name
string | null
​
topic_ids
​
config
TransformConfig · object
​
implementation
TransformsMapFilterImplementation · object
​
metrics
TransformMetric · object
​
live_version
string | null
​
preview_version
string | null
​
version
integer
default:1
​
updated_timestamp
string<date-time> | null
​
implementation_updated_timestamp
string<date-time> | null
​
description
string | null
​
history_created_timestamp
string<date-time> | null
​
history_updated_timestamp
string<date-time> | null