TypeScript Development Kit

Build Streamkap transforms using TypeScript with your favorite npm packages and full testing support.

Repository: streamkap-com/transform-kit-typescript

Quick Start

Prerequisites: Node.js 16+, npm 8+

git clone https://github.com/streamkap-com/transform-kit-typescript.git
cd transform-kit-typescript
npm install          # Install dependencies
npm run build        # Build transforms
npm test            # Run tests

Note: You may see some npm warnings during install - these don't affect functionality.

Supported Transform Types

How to Use

1. Define Your Data Structure

Tell the system what your data looks like by editing the TypeScript files:

// src/OrderType1.ts
export interface OrderType1 {
    _id: string;
    order_number: number;
    total_amount: number;
}

// src/MergedOrder.ts
export interface MergedOrder {
    _id: string;
    processed_at: string;
    formatted_total: string;
}

2. Write Your Business Logic

Open src/OrderTransformer.ts and write the code that processes your data:

export class OrderTransformer {
    transform(input: OrderType1): MergedOrder {
        const moment = require('moment');
        
        return {
            _id: input._id,
            processed_at: moment().toISOString(),
            formatted_total: `$${input.total_amount.toFixed(2)}`
        };
    }
}

3. Add External Libraries (Optional)

You can use popular JavaScript libraries to help with your transforms:

npm install moment lodash uuid  # These are already installed in the project

What works: JavaScript libraries like moment (dates), lodash (utilities), uuid What doesn't work: Libraries that require system-level access or Node.js-specific features

4. Build and Test

npm run build  # Generates JavaScript files
npm test       # Runs all tests

Files are generated in the transforms/ directory.

5. Deploy to Streamkap

1. Create your transform in the Streamkap web interface
2. Go to the Implementation tab
3. Copy the entire contents of your generated file:
   • For value transforms: value_transform.js
   • For key transforms: key_transform.js
   • For topic transforms: topic_transform.js
4. Paste into Streamkap's code editor (this replaces the default code)
5. Save and Deploy - your TypeScript code is now running!

The files are completely self-contained, so just copy the entire file content.

Generated Files

transforms/
├── map-filter/
│   ├── value_transform.js       # For value transforms
│   ├── key_transform.js         # For key transforms  
│   └── mapFilterTransform.js    # Combined version
├── fan-out/
│   ├── value_transform.js
│   ├── topic_transform.js
│   └── fanOutTransform.js       # Combined version
├── enrich-async/
│   ├── value_transform.js
│   └── enrichAsyncTransform.js  # Combined version
└── un-nesting/
    ├── value_transform.js
    └── unNestingTransform.js    # Combined version

Each file is self-contained with all npm dependencies bundled inside.

Real-World Examples

These are just examples - replace the data types and logic with your own use case.

Order Processing

transform(input: Order): EnrichedOrder {
    const moment = require('moment');
    
    return {
        _id: input._id,
        processed_at: moment().format(),
        priority: input.total > 100 ? 'high' : 'normal'
    };
}

Sensor Data

transform(input: SensorReading): ProcessedSensor {
    const _ = require('lodash');
    
    return {
        sensor_id: input.sensor_id,
        average: _.meanBy(input.readings, 'value'),
        count: input.readings.length
    };
}

Order Processing (Actual Implementation)

transformOrderType1(inputOrder: OrderType1): MergedOrder {
    const now = moment();
    
    // Example using lodash for data manipulation
    const cleanedOrder = _.omitBy(inputOrder, _.isUndefined);
    const hasValidCustomer = _.has(cleanedOrder, 'customer.name') && !_.isEmpty(cleanedOrder.customer.name);
    
    // Example using uuid for unique identifiers
    const processingId = uuidv4();
    
    return {
        version: '0.1.4',
        _id: inputOrder._id,
        order_number: inputOrder.order_number,
        processed_at: now.toISOString(),
        processed_time: now.format('YYYY-MM-DD HH:mm:ss'),
        processing_id: processingId, // uuid generated
        has_valid_customer: hasValidCustomer, // lodash validation
        field_count: _.keys(cleanedOrder).length, // lodash utility
        // ... other fields
    };
}

Key Files You'll Work With

src/
├── OrderTransformer.ts     # Your main business logic goes here
├── OrderType1.ts          # Define what your input data looks like
├── MergedOrder.ts         # Define what your output data looks like
├── *.test.ts             # Automated tests (optional to modify)
└── index.ts              # System file (don't modify)

transforms/               # Generated files (copy these to Streamkap)

What You Get

• Type Safety: Catch errors before deployment with TypeScript
• Rich Ecosystem: Use thousands of npm packages in your transforms
• Testing: Automated tests ensure your code works correctly
• Self-Contained: Generated files include everything needed - no external dependencies
• All Transform Types: Works with every Streamkap transform type
• Standard Workflow: Familiar git-based development process

Troubleshooting

Build fails?

npm install
npm run build

Tests fail?

npm run build  # Always build first
npm test

Streamkap deployment not working?

• Make sure you copy the entire file contents (including all the bundled code)
• Always run npm test locally before deploying
• Check the Streamkap error console for specific error messages

⚠️

Always run npm run build before npm test. The tests check the actual files that get deployed to Streamkap.

Need More Details?

This guide covers the basics. For advanced usage, troubleshooting, and detailed explanations, check the README.md file in the repository.