> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streamkap.com/llms.txt
> Use this file to discover all available pages before exploring further.

# TypeScript Development Kit

> Development environment for building TypeScript transforms with npm dependencies, testing, and bundling

Build Streamkap transforms using TypeScript with your favorite npm packages and full testing support. Use as a standalone project or integrate into your existing codebase.

**Repository**: [streamkap-com/transform-kit-typescript](https://github.com/streamkap-com/transform-kit-typescript)

## Choose Your Path

### 🆕 **New Project** (Start from scratch)

Use the entire repository as your development environment

### 🔄 **Existing Project** (Add to your codebase)

Add TypeScript transform capabilities to your current project

## Quick Start - New Project

**Prerequisites**: Node.js 16+, npm 8+

<CodeGroup>
  ```bash Terminal theme={null}
  git clone https://github.com/streamkap-com/transform-kit-typescript.git
  cd transform-kit-typescript
  npm install && npm run build && npm test
  ls -la transforms/  # ← Copy these .js files to Streamkap
  ```
</CodeGroup>

*Note: You may see some npm warnings during install - these don't affect functionality.*

📋 **Next**: See [Which File to Use?](#-which-file-to-use) and [deployment steps](#5-deploy-to-streamkap) below.

## Supported Transform Types

| Transform Type     | Files Generated                          | Description                      |
| ------------------ | ---------------------------------------- | -------------------------------- |
| **Map/Filter**     | `valueTransform.js`, `keyTransform.js`   | Transform and filter records     |
| **Fan Out**        | `valueTransform.js`, `topicTransform.js` | Route records to multiple topics |
| **Enrich (Async)** | `valueTransform.js`                      | Async data enrichment            |
| **Un-nesting**     | `valueTransform.js`                      | Flatten nested objects           |

## How to Use - New Project

### 1. Define Your Data Structure

Tell the system what your data looks like by editing the TypeScript files:

<CodeGroup>
  ```typescript TypeScript theme={null}
  // src/OrderType1.ts
  export interface OrderType1 {
      _id: string;
      order_number: number;
      total_amount: number;
  }

  // src/MergedOrder.ts
  export interface MergedOrder {
      _id: string;
      processed_at: string;
      formatted_total: string;
  }
  ```
</CodeGroup>

### 2. Write Your Business Logic

Open `src/OrderTransformer.ts` and write the code that processes your data:

<CodeGroup>
  ```typescript TypeScript theme={null}
  export class OrderTransformer {
      transform(input: OrderType1): MergedOrder {
          const moment = require('moment');
          
          return {
              _id: input._id,
              processed_at: moment().toISOString(),
              formatted_total: `$${input.total_amount.toFixed(2)}`
          };
      }
  }
  ```
</CodeGroup>

### 3. Add External Libraries (Optional)

You can use popular JavaScript libraries to help with your transforms:

<CodeGroup>
  ```bash Terminal theme={null}
  npm install moment lodash uuid  # These are already installed in the project
  ```
</CodeGroup>

**What works**: JavaScript libraries like moment (dates), lodash (utilities), uuid
**What doesn't work**: Libraries that require system-level access or Node.js-specific features

### 4. Build and Test

<CodeGroup>
  ```bash Terminal theme={null}
  npm run build  # Generates JavaScript files
  npm test       # Runs all tests
  ```
</CodeGroup>

Files are generated in the `transforms/` directory.

### 5. Deploy to Streamkap

**Coming soon:** deploy using CLI , deploy using Terraform.

1. **Create your transform** in the Streamkap web interface
2. **Go to the Implementation tab**
3. **Copy the entire contents** of your generated file:
   * **Always required**: `valueTransform.js` (your main business logic)
   * **Optional**: `keyTransform.js` (only if you need custom partitioning - leave blank otherwise)
   * **Optional**: `topicTransform.js` (only for fan-out routing - leave blank otherwise)
4. **Paste into Streamkap's code editor** (this replaces the default code)
5. **Save and Deploy** - your TypeScript code is now running!

*The files are completely self-contained, so just copy the entire file content.*

### 🤔 Which File to Use?

| Transform Type    | File to Copy                       | Purpose                                             |
| ----------------- | ---------------------------------- | --------------------------------------------------- |
| **map\_filter**   | `valueTransform.js` (**required**) | Transform and filter records                        |
|                   | `keyTransform.js` (*optional*)     | Leave blank in Streamkap if no key changes needed   |
| **fan\_out**      | `valueTransform.js` (**required**) | Transform records for routing                       |
|                   | `topicTransform.js` (*optional*)   | Leave blank in Streamkap if no topic changes needed |
| **enrich\_async** | `valueTransform.js` (**required**) | Enrich with external APIs                           |
| **un\_nesting**   | `valueTransform.js` (**required**) | Flatten nested objects                              |

💡 **Key Points**:

* **`valueTransform.js`** is **always required** (contains your main business logic)
* **`keyTransform.js`** is **optional** - only needed for custom partitioning logic; leave blank in Streamkap if you want to keep the original message key
* **`topicTransform.js`** is **optional** - only needed for fan-out transforms to route messages to different topics; leave blank in Streamkap if sending to the original topic

## Generated Files

```
transforms/
├── map-filter/
│   ├── valueTransform.js
│   └── keyTransform.js
├── fan-out/
│   ├── valueTransform.js
│   └── topicTransform.js
├── enrich-async/
│   └── valueTransform.js
└── un-nesting/
    └── valueTransform.js
```

Each file is self-contained with all npm dependencies bundled inside.

***

## How to Use - Existing Project

Add Streamkap bundler to your existing codebase in 3 steps:

### 1. Copy Files

<CodeGroup>
  ```bash Terminal theme={null}
  # Copy bundler and transform files
  cp build-multiple.js your-project/
  cp test-selective.js your-project/
  mkdir -p your-project/src/
  cp src/value_transform.ts your-project/src/
  cp src/key_transform.ts your-project/src/
  cp src/topic_transform.ts your-project/src/
  # Optionally copy the example interfaces and transformer as reference
  cp src/OrderTransformer.ts your-project/src/ 
  cp src/Customer.ts your-project/src/
  cp src/OrderType1.ts your-project/src/
  cp src/OrderType2.ts your-project/src/
  cp src/MergedOrder.ts your-project/src/
  ```
</CodeGroup>

### 2. Update package.json

<CodeGroup>
  ```json JSON theme={null}
  {
    "scripts": {
      "build": "node build-multiple.js --all",
      "build:map-filter": "node build-multiple.js --map-filter",
      "build:fan-out": "node build-multiple.js --fan-out",
      "build:enrich-async": "node build-multiple.js --enrich-async",
      "build:un-nesting": "node build-multiple.js --un-nesting",
      "test": "node test-selective.js",
      "test:map-filter": "npm run build:map-filter && node test-selective.js --map-filter"
    },
    "dependencies": {
      "lodash": "^4.17.21",
      "moment": "^2.30.1", 
      "uuid": "^11.1.0"
    },
    "devDependencies": {
      "@types/lodash": "^4.17.20",
      "@types/moment": "^2.11.29",
      "@types/uuid": "^10.0.0",
      "esbuild": "^0.8.27",
      "typescript": "^4.0.0"
    }
  }
  ```
</CodeGroup>

### 3. Connect Your Logic

Create your own OrderTransformer or update the existing one:

<CodeGroup>
  ```typescript TypeScript theme={null}
  // src/YourTransformer.ts
  import { YourService } from './your-existing-logic';

  export class YourTransformer {
      private yourService = new YourService();
      
      public transform(input: any): any {
          // Use your existing business logic
          return this.yourService.processData(input);
      }
  }
  ```
</CodeGroup>

Then update the transform files to use your transformer:

<CodeGroup>
  ```typescript TypeScript theme={null}
  // src/value_transform.ts
  import { YourTransformer } from "./YourTransformer";

  export function _streamkap_transform(valueObject: any, keyObject: any, topic: any, timestamp: any) {
      var transformer = new YourTransformer();
      var transformedRecord = transformer.transform(valueObject);
      return transformedRecord;
  }
  ```
</CodeGroup>

### 4. Build & Deploy

<CodeGroup>
  ```bash Terminal theme={null}
  # Install dependencies
  npm install

  # Build all transforms
  npm run build

  # OR build specific transforms only
  npm run build:map-filter                            # Build specific type
  node build-multiple.js --map-filter --enrich-async  # Build multiple types
  node build-multiple.js --all                        # Build all transforms

  # Check generated files
  ls transforms/  # Copy these .js files to Streamkap
  ```
</CodeGroup>

**That's it!** Your transforms are generated in `transforms/` folder.

📋 **Next**: See [Which File to Use?](#-which-file-to-use) below for deployment guidance.

***

## Real-World Examples

*These are just examples - replace the data types and logic with your own use case.*

### Order Processing

<CodeGroup>
  ```typescript TypeScript theme={null}
  transform(input: Order): EnrichedOrder {
      const moment = require('moment');
      
      return {
          _id: input._id,
          processed_at: moment().format(),
          priority: input.total > 100 ? 'high' : 'normal'
      };
  }
  ```
</CodeGroup>

### Sensor Data

<CodeGroup>
  ```typescript TypeScript theme={null}
  transform(input: SensorReading): ProcessedSensor {
      const _ = require('lodash');
      
      return {
          sensor_id: input.sensor_id,
          average: _.meanBy(input.readings, 'value'),
          count: input.readings.length
      };
  }
  ```
</CodeGroup>

### Order Processing (Actual Implementation)

<CodeGroup>
  ```typescript TypeScript theme={null}
  transformOrderType1(inputOrder: OrderType1): MergedOrder {
      const now = moment();
      
      // Example using lodash for data manipulation
      const cleanedOrder = _.omitBy(inputOrder, _.isUndefined);
      const hasValidCustomer = _.has(cleanedOrder, 'customer.name') && !_.isEmpty(cleanedOrder.customer.name);
      
      // Example using uuid for unique identifiers
      const processingId = uuidv4();
      
      return {
          version: '0.1.4',
          _id: inputOrder._id,
          order_number: inputOrder.order_number,
          processed_at: now.toISOString(),
          processed_time: now.format('YYYY-MM-DD HH:mm:ss'),
          processing_id: processingId, // uuid generated
          has_valid_customer: hasValidCustomer, // lodash validation
          field_count: _.keys(cleanedOrder).length, // lodash utility
          // ... other fields
      };
  }
  ```
</CodeGroup>

## Key Files You'll Work With

**New Project:**

```
src/
├── OrderTransformer.ts     # Your main business logic goes here
├── OrderType1.ts          # Define what your input data looks like
├── MergedOrder.ts         # Define what your output data looks like
├── *.test.ts             # Automated tests (optional to modify)
└── index.ts              # System file (don't modify)
```

**Existing Project:**

```
src/templates/
├── commonTransform.ts     # Connect your existing services here
├── valueTransform.ts      # Main value transformation logic
├── keyTransform.ts        # Key transformation for partitioning
├── topicTransform.ts      # Topic routing for fan-out
└── index.ts              # System file (don't modify)
```

**Both approaches generate:**

```
transforms/               # Generated files (copy these to Streamkap)
```

## What You Get

* **Type Safety**: Catch errors before deployment with TypeScript
* **Rich Ecosystem**: Use thousands of npm packages in your transforms
* **Testing**: Automated tests ensure your code works correctly
* **Self-Contained**: Generated files include everything needed - no external dependencies
* **All Transform Types**: Works with every Streamkap transform type
* **Two Integration Options**: Use as standalone project or integrate into existing codebase
* **Standard Workflow**: Familiar git-based development process

## Troubleshooting

**Tests failing?**

<CodeGroup>
  ```bash Terminal theme={null}
  npm run build  # Build first - tests validate generated files
  npm test -- --verbose
  ```
</CodeGroup>

**Build errors?**

<CodeGroup>
  ```bash Terminal theme={null}
  npm install
  npm run build 2>&1 | grep -i error
  ```
</CodeGroup>

**Only need specific transforms?**

<CodeGroup>
  ```bash Terminal theme={null}
  # Build individual transform types for faster development
  npm run build:map-filter      # Only builds map/filter transforms
  npm run test:enrich-async     # Builds and tests async enrichment

  # Test without building (uses existing builds)
  npm test -- --map-filter     # Test only map/filter transforms
  npm test -- --fan-out --un-nesting  # Test multiple specific types

  # Build multiple specific types
  node build-multiple.js --fan-out --un-nesting

  # Available types: --map-filter, --fan-out, --enrich-async, --un-nesting, --all
  ```
</CodeGroup>

**Streamkap deployment not working?**

* Make sure you copy the **entire file contents** (including all the bundled code)
* Always run `npm test` locally before deploying
* Check the Streamkap error console for specific error messages

<Warning>
  Always run `npm run build` before `npm test`. The tests check the actual files that get deployed to Streamkap.
</Warning>

## Need More Details?

This guide covers the basics. For advanced usage, troubleshooting, and detailed explanations, check the README.md file in the repository.