Skip to main content
A Streamkap service is the complete infrastructure stack for your Change Data Capture (CDC) pipelines, including Apache Kafka, Apache Flink, Debezium, connectors, and supporting components. Services are isolated environments that can represent different stages (Development, Staging, Production) or geographic regions.

​
Overview

The Services page displays all services in your organization with their current status:

​
Service Architecture

Each Streamkap service includes:
  • Pipelines: pipelines streaming source topics and/or transformed topics to destinations
  • Source Connectors: Database CDC connectors (MySQL, PostgreSQL, DynamoDB, etc.)
  • Destination Connectors: Data warehouse/lake connectors (Snowflake, BigQuery, ClickHouse, etc.)
  • Transforms: Data transformation engine
  • Topics: Kafka topics for data streams
  • Monitoring: Metrics, logs, and alerts infrastructure
All components are configured for high performance, scalability, and reliability.

​
Service States

​
AVAILABLE

The service is fully operational and ready to use:
  • All infrastructure components are running
  • Connectors can be created and managed
  • Data pipelines can process events
  • Marked with green badge
  • Shows CURRENT if you’re actively using this service

​
UNAVAILABLE

The service exists but is not currently operational:
  • Infrastructure is paused or stopped
  • Cannot create or modify connectors
  • Data pipelines are not processing
  • Use “Switch to this” button to activate if needed
  • Marked with gray badge
Services in UNAVAILABLE state do not incur compute charges, but may retain data depending on your configuration.

​
Services Table

The table displays the following information for each service:
  • Name: Service identifier
    • Shows CURRENT badge for the active service
    • Shows “Switch to this” button for inactive services
  • Type: Service environment type (e.g., production, development)
  • Created On: Timestamp when the service was provisioned
  • State: Current operational status (AVAILABLE, UNAVAILABLE)
  • Cloud: Cloud provider (AWS, Azure, GCP)
  • Region: Geographic region (e.g., us-west-2, us-east-1, ap-southeast-2)
  • Actions: Three-dot menu for service operations
Use the search bar to filter services by name:
  • Type to filter the service list in real-time
  • Useful when managing many services across environments

​
Switching Between Services

To switch from one service to another:
1

Identify Target Service

Locate the service you want to switch to in the services table.
2

Click Switch to this

Click the “Switch to this” button next to the service name.
3

Wait for Switch

The page will reload and you’ll be redirected to the selected service.
4

Verify Current Service

Check the sidebar header to confirm you’re in the correct service.
Switching services changes the entire context of the application. All pages (Pipelines, Connectors, Topics, etc.) will now show data for the selected service.

​
Service Actions Menu

Click the three-dot menu on any service row to access service-specific actions:

​
Copy ID

Copy the unique service identifier to your clipboard:
  • Use Case: API integrations, support requests, automation scripts
  • Format: UUID string
  • Example: a1b2c3d4-e5f6-7890-abcd-ef1234567890

​
Copy Kafka Brokers

Copy the Kafka broker connection string:
  • Use Case: Direct Kafka client connections, external applications
  • Format: Comma-separated broker list with ports
  • Example: broker1.streamkap.com:9092,broker2.streamkap.com:9092,broker3.streamkap.com:9092
  • Security: Requires authentication credentials from Kafka Access page
Use this connection string to connect external Kafka clients to your Streamkap Kafka cluster. Combine with credentials from Kafka Access for authentication.

​
API Tokens

Navigate to the API Tokens management page:
  • Purpose: Generate and manage API tokens for programmatic access
  • Use Case: CI/CD pipelines, automation, infrastructure as code
  • Scope: Tokens are service-specific
  • Permissions: Control access to API endpoints
API tokens allow you to automate service operations via the Streamkap REST API. See the API Reference for available endpoints.

​
Common Use Cases

​
Environment Separation

Maintain separate services for different environments: Development Service:
  • Test schema changes and new connectors
  • Experiment with transforms and pipelines
  • Lower resource allocation, smaller dataset
  • Region: Choose closest to development team
Staging Service:
  • Mirror production configuration
  • Run integration tests
  • Validate deployments before production
  • Same region as production for realistic testing
Production Service:
  • Live data pipelines
  • High availability configuration
  • Optimal region for data sources
  • Monitoring and alerting enabled

​
Geographic Distribution

Create services in multiple regions for: Data Locality:
  • Reduce latency by placing services near data sources
  • Comply with data residency regulations (GDPR, etc.)
  • Example: EU service for European databases, US service for American databases
Disaster Recovery:
  • Replicate pipelines across regions
  • Failover capability for business continuity
  • Active-active or active-passive configurations
Global Operations:
  • Support teams in different time zones
  • Reduce network latency for distributed teams
  • Scale horizontally across regions

​
Creating New Services

Service creation is managed by Streamkap support to ensure proper infrastructure provisioning and configuration.
To request a new service:
1

Contact Streamkap Support

Reach out via the Contact Us button in the sidebar or email [email protected].
2

Provide Service Requirements

Specify the following:
  • Environment type: Production, Development, Staging, etc.
  • Cloud provider: AWS, Azure, or GCP
  • Region: Geographic location (e.g., us-west-2, eu-central-1)
  • Expected throughput: Data volume and connector count
  • Special requirements: VPC peering, private link, compliance needs
3

Service Provisioning

Streamkap will provision the infrastructure:
  • Kafka cluster setup
  • Flink deployment
  • Networking configuration
  • Monitoring setup
  • Typically takes 1-2 business days
4

Access Confirmation

You’ll receive confirmation when the service is ready, and it will appear in your Services list.

​
Managing Service Resources

Each service has independent resources:

​
Connectors

Sources and destinations are service-specific:
  • Must be recreated in each service
  • Cannot be moved between services
  • Configuration can be exported/imported

​
Pipelines

Data pipelines exist within a single service:
  • Define data flow from source to destination
  • Cannot span multiple services
  • Must be recreated in each environment

​
Topics

Kafka topics are isolated to each service:
  • Topic names can be reused across services
  • Data is not shared between services
  • Replication limited to service boundaries

​
Billing

Usage is tracked per service:
  • View service-specific usage in Billing page
  • Switch services to see different usage metrics
  • Services are billed independently

​
Best Practices

  1. Use Consistent Naming: Name services clearly (e.g., “Production-US-East”, “Development-EU”)
  2. Document Service Purpose: Maintain documentation of what each service is for
  3. Limit Production Access: Restrict who can switch to production services
  4. Test in Non-Production First: Always validate changes in dev/staging before production
  5. Monitor Service Health: Regularly check service state and resource usage
  6. Copy IDs for Automation: Store service IDs in your infrastructure as code
  7. Keep Kafka Brokers Secure: Treat broker connection strings as sensitive data
  8. Rotate API Tokens: Regularly rotate tokens for security
  9. Plan for Multi-Region: Consider geographic distribution for global operations
  10. Align Regions with Data: Place services near your data sources to reduce latency

​
Troubleshooting

​
Service Shows UNAVAILABLE

If a service is unexpectedly unavailable:
  1. Check Service Status: Contact Streamkap support for infrastructure status
  2. Verify Billing: Ensure account is in good standing
  3. Review Recent Changes: Check if service was intentionally paused
  4. Check Maintenance Window: Service may be undergoing scheduled maintenance

​
Cannot Switch Services

If the “Switch to this” button doesn’t work:
  1. Refresh Page: Reload the browser page
  2. Clear Cache: Clear browser cache and cookies
  3. Check Permissions: Verify you have access to the target service
  4. Contact Support: Reach out if the issue persists

​
Kafka Brokers Not Connecting

If external clients can’t connect using copied broker string:
  1. Verify Credentials: Ensure you have valid Kafka Access credentials
  2. Check Network: Verify network connectivity and firewall rules
  3. Confirm Service State: Service must be AVAILABLE for connections
  4. Review Security: Check SSL/TLS configuration if required
  5. Use Correct Port: Verify you’re using the correct port (usually 9092 or 9093 for SSL)

​
API Token Not Working

If API tokens don’t authenticate:
  1. Verify Token Scope: Ensure token is for the correct service
  2. Check Expiration: Tokens may have expiration dates
  3. Confirm Permissions: Token must have required permissions for the operation
  4. Regenerate Token: Create a new token if issues persist
  • Kafka Access - Manage Kafka users and obtain credentials for broker connections
  • Pipelines - Create and manage data pipelines within a service
  • Sources - Configure source connectors in your service
  • Destinations - Configure destination connectors in your service
  • Billing - View per-service usage and billing information
  • API Reference - Use API tokens for programmatic service management
⌘I