Test Run Agent
Run an agent against a single sample record without deploying to Flink.
Synchronous, hard-capped at 15s wall-clock. Calls the LLM provider directly + dispatches MCP and HTTP tools live; transform tools and memory are intentionally not exercised (the Flink runtime owns them). Per-tenant rate limit: shares the validate-llm bucket (30 req/min/tenant) -> 429.
Provider failures (timeouts, schema errors, unsupported provider) land
as a structured error object on the envelope (HTTP still 200) so
the FE can render the partial timeline alongside the error context.
Only the input-token cap raises (422) - at that point we can’t even
start the run.
/agents/test-runAuthorizationBearer token · headerrequiredapplication/jsonconfigCreateAgentConfigRequestrequiredRequest body for POST /agents/config - create a config-based agent.
Show propertiesHide properties
namestringrequiredAgent display name
descriptionstring | anyHuman-readable context for the agent
Show propertiesHide properties
stringanyenabledbooleanKill switch. When false, the Flink runtime accepts the deploy but stops emitting output — used by support to disable a runaway tenant agent without a full cancel/redeploy.
agentTypeAgentTypeEnumAgent type — derived from the tool list post-validation; client-supplied value is overwritten
workflowreactinputAgentInputConfigrequiredInput configuration for the agent.
Show propertiesHide properties
topicPatternstringrequiredRegex pattern for input topics, e.g. '^(orders)$'
inputSerializationAgentSerializationEnumInput deserialization format
JsonConfluentAvroConfluentfieldsstring[] | anyColumn-level filter - only these fields sent to LLM
Show propertiesHide properties
stringstringanycreateTableSQLstring | anyFlink SQL CREATE TABLE (required if filterSQL is set)
Show propertiesHide properties
stringanyfilterSQLstring | anyWHERE clause predicate for row-level filtering
Show propertiesHide properties
stringanyoutputAgentOutputConfigrequiredOutput configuration for the agent.
Show propertiesHide properties
topicstringrequiredFixed output topic name
deadLetterTopicstring | anyDLQ topic name
Show propertiesHide properties
stringanyoutputSerializationAgentSerializationEnumOutput serialization format
JsonConfluentAvroConfluentschemaobjectOutput schema: {field_name: TYPE_STRING}
llmAgentLlmConfigrequiredInline LLM configuration embedded on the agent.
Carries provider, credentials, model, and tuning controls as a single
blob. The FE prefills the form from a saved :class:AgentLlmConnection
(which now carries default model + tuning + reasoning controls) and
the user can override per-agent before save; the resulting blob is
stored verbatim and shipped to the Flink runtime at deploy.
Two credential shapes (XOR, mirrors :class:ValidateLlmRequest):
- Inline -
apiKeyis filled directly. The BE KMS-encrypts it at save and decrypts it at deploy / test-run. - Linked -
llmConnectionIdreferences a row inagent_connections.llmConnectionsandapiKeyis left empty. The BE resolves the real key server-side at deploy and at test-run via :func:agents_service.resolve_saved_llm_credentials, so the browser never needs to hold (or re-paste) the stored plaintext when editing an existing agent.
provider == ollama accepts an empty key without a link (the
runtime is local and unauthenticated).
Show propertiesHide properties
providerAgentLlmProviderEnumrequiredUnified LLM provider enum.
A single AgentLlmConnection row carries one provider and a set of
capabilities (chat / embedding). PROVIDER_CAPABILITIES below pins
which capabilities each provider can serve — picked by the FE Connections
drawer and re-validated server-side on every write.
anthropicopenaiopenai-responsesollamaazureazure-openaibedrockqwenopenai-compatiblemodelstringrequiredModel name, e.g. claude-sonnet-4-20250514
apiKeystringAPI key - use '${SECRET:ENV_VAR_NAME}' for env var resolution. Empty when llmConnectionId is set.
llmConnectionIdstring | anyTenant-scoped reference to agent_connections.llmConnections[].id. When set with an empty apiKey, the BE resolves the real key server-side at deploy and at test-run.
Show propertiesHide properties
stringanybaseUrlstring | anyBase URL for OpenAI-compatible proxies
Show propertiesHide properties
stringanytemperaturenumber | anySampling temperature (null = matrix-skipped)
Show propertiesHide properties
numberanymaxTokensinteger | anyMax output tokens (null = matrix-skipped)
Show propertiesHide properties
integeranytimeoutinteger | anyRequest timeout in seconds (null = matrix-skipped)
Show propertiesHide properties
integeranyreasoningEffortstring | anyOpenAI / GPT-5 reasoning effort. The accepted set is per-model: the matrix in app/utils/llm_capabilities.py rejects values not in caps.reasoning_effort_values for the chosen provider+model.
Show propertiesHide properties
stringanythinkingBudgetTokensinteger | anyAnthropic Claude 4 extended-thinking budget (tokens). Non-null enables thinking.
Show propertiesHide properties
integeranyollamaThinkboolean | anyOllama 'think' toggle for reasoning models
Show propertiesHide properties
booleananymaxRetriesinteger | anyUniversal — max retry attempts on LLM call. Accepted by every provider.
Show propertiesHide properties
integeranyregionstring | anyBedrock: AWS region (e.g. us-east-1). Ignored for non-bedrock providers.
Show propertiesHide properties
stringanystrictboolean | anyopenai-responses: enable JSON-schema strict mode.
Show propertiesHide properties
booleananystoreboolean | anyopenai-responses: server-side response storage flag.
Show propertiesHide properties
booleananyinstructionsstring | anyopenai-responses: system-level instructions passed as a top-level Responses param.
Show propertiesHide properties
stringanyadditionalKwargsobject | anyopenai-responses / azure-openai: free-form extra request params forwarded verbatim by the runtime.
Show propertiesHide properties
objectanyapiVersionstring | anyazure-openai: Azure OpenAI API version (e.g. '2024-02-01'). Required for azure-openai.
Show propertiesHide properties
stringanyazureEndpointstring | anyazure-openai: Azure resource endpoint (e.g. https://<resource>.openai.azure.com). Required.
Show propertiesHide properties
stringanyazureUrlPathModestring | anyazure-openai: URL path resolution — AUTO / LEGACY / UNIFIED. Optional.
Show propertiesHide properties
stringanypromptsAgentPromptConfigrequiredPrompt configuration for the agent.
The final system prompt is auto-assembled by build_system_prompt() from the
agent type template + output schema + tool descriptions + custom_instructions.
The system field stores the assembled result (set by the backend, not the user).
Show propertiesHide properties
systemstringAssembled system prompt (auto-generated by backend)
customInstructionsstringUser-provided instructions appended to the auto-generated prompt
userstringUser prompt template - {input_json} is replaced with the input record
mcpServerAgentMcpServerConfig | anyShared MCP server config for all MCP tools
Show propertiesHide properties
projectKeyIdstring | anyID of an agentic-enabled Project Key. When set, deploy-time resolves the Streamkap MCP serverUrl + auth header from the PK row. Mutually exclusive with serverUrl - one or the other, never both.
Show propertiesHide properties
stringanyserverUrlstring | anyExternal MCP server URL. Required when projectKeyId is unset. Server-stamped from the PK row when projectKeyId is set.
Show propertiesHide properties
stringanyheadersobjectAuth headers for MCP server
anytoolsAgentToolConfig[]Agent tools (MCP, HTTP, transform)
Show propertiesHide properties
AgentToolConfignamestringrequiredTool name (must be unique)
typeAgentToolTypeEnumrequiredTool type: http, transform, or mcp
httptransformmcpdescriptionstringTool description for the LLM
configobjectType-specific configuration
parametersAgentToolParameterConfig[]Tool parameters the LLM can provide
Show propertiesHide properties
AgentToolParameterConfignamestringrequiredParameter name
typestringParameter type (string, number, boolean)
descriptionstringParameter description for the LLM
memoryAgentMemoryConfig | anyMemory config (short-term + long-term)
Show propertiesHide properties
keyFieldstring | anyField to partition memory by (e.g. 'customer_id'). Default: topic key
Show propertiesHide properties
stringanyshortTermAgentShortTermMemoryConfigShort-term (conversation) memory config.
Show propertiesHide properties
enabledbooleanEnable short-term memory
ttlMsintegerTime-to-live in milliseconds (default: 1 hour)
maxEntriesintegerMax conversation entries per key
longTermAgentLongTermMemoryConfigLong-term (vector store) memory config.
References a saved AgentVectorStoreConnection by id. At deploy time
the resolver reads the connection's apiKey + endpoint and composes
the inline vectorStore block the Java runtime expects.
Legacy agents may still carry destinationId (a Pinecone destination
connector from the old flow). The deploy resolver handles both: if
vectorStoreConnectionId is set it wins; otherwise it falls back to
the legacy destinationId path. New agents created through the FE
always use vectorStoreConnectionId.
Show propertiesHide properties
enabledbooleanEnable long-term memory
vectorStoreConnectionIdstring | anyReference to a saved AgentVectorStoreConnection. Resolved at deploy time.
Show propertiesHide properties
stringanynamespacestring | anyPer-agent namespace override (falls back to connection's defaultNamespace)
Show propertiesHide properties
stringanydestinationIdstring | any(Legacy) ObjectId of a Pinecone destination connector. Prefer vectorStoreConnectionId.
Show propertiesHide properties
stringanyanyknowledgeBasesAgentKnowledgeBaseRef[]Knowledge bases this agent can query at runtime for RAG
Show propertiesHide properties
AgentKnowledgeBaseRefidstringrequiredKnowledge base entity ID
namestringDisplay name (denormalized for UI)
processingAgentProcessingConfigProcessing configuration for the agent.
Show propertiesHide properties
parallelismintegerParallelism
checkpointIntervalMinintegerCheckpoint interval in minutes
maxIterationsintegerMax tool-call iterations per record (react type only)
maxTokensPerHourintegerPer-TaskManager rolling-hour token budget. 0 = unlimited. When exceeded, records pass through with a _budget_exceeded: true marker.
sampleRecordobjectrequiredOne Kafka-shaped record to feed the agent
agentIdstring | anyResolves an empty / masked apiKey against the stored agent's encrypted blob. Post-cutover the wizard sends apiKey: '' for every existing-agent test-run; this lets the BE swap in the stored key for legacy inline-key agents that were saved before the Connection cutover (no llmConnectionId to resolve via).
Show propertiesHide properties
stringanysavedConnectionIdstring | anyMirror of the ValidateLlmRequest.savedConnectionId path.
Show propertiesHide properties
stringanyskipToolsbooleanWhen true, every MCP / HTTP tool dispatch returns a canned {skipped: true} payload to the LLM. Use to iterate on prompt shape without burning real tool calls.
Successful Response
stepsTestRunStep[]requiredShow propertiesHide properties
TestRunStepkindstringrequiredllmtoolmodelstring | anyShow propertiesHide properties
stringanytokensIninteger | anyShow propertiesHide properties
integeranytokensOutinteger | anyShow propertiesHide properties
integeranystopReasonstring | anyShow propertiesHide properties
stringanytextstring | anyShow propertiesHide properties
stringanytoolCallsobject[] | anyShow propertiesHide properties
objectobjectanytoolNamestring | anyShow propertiesHide properties
stringanytoolTypestring | anyShow propertiesHide properties
stringanyrequestobject | anyShow propertiesHide properties
objectanyresultobject | anyShow propertiesHide properties
objectanyskippedboolean | anyShow propertiesHide properties
booleananyskippedReasonstring | anyShow propertiesHide properties
stringanydurationMsinteger | anyShow propertiesHide properties
integeranyerrorstring | anyShow propertiesHide properties
stringanyfinalOutputanystoppedReasonstringrequiredcompletedmax_iterationserrortimeoutiterationsUsedintegerrequiredtokensInintegerrequiredtokensOutintegerrequireddurationMsintegerrequiredapproximationsstring[]Free-form list of caveats, e.g. 'memory_excluded', 'transforms_skipped'.
suggestionsTestRunSuggestion[]Deterministic hints (max 4) computed from the agent config + run result. Empty when the run produced a useful structured output. Each suggestion has a stable kind the FE can map to icon / copy.
Show propertiesHide properties
TestRunSuggestionkindstringrequiredmissing_output_schematools_not_invokednear_empty_responseskip_instruction_may_applymax_tokens_truncatedmessagestringrequiredwarningsTestRunWarning[]Static config-vs-record checks (e.g. memory keyField missing on the sample) that the Flink runtime would silently swallow at deploy. Empty when nothing is amiss.
Show propertiesHide properties
TestRunWarningkindstringrequiredmemory_key_field_missinginput_fields_not_in_recordmax_tokens_clampedmessagestringrequiredoutputSchemaCheckTestRunSchemaCheck | anyField-level diff between the declared output.schema and the LLM's finalOutput. Null when no schema was declared OR the run errored / produced a non-structured (text envelope) result. ok=true means every declared field is present with a compatible type.
Show propertiesHide properties
okbooleanrequiredmissingstring[]Fields declared in schema but absent from finalOutput
extrastring[]Fields present in finalOutput but not declared
typeMismatchesTestRunSchemaTypeMismatch[]Fields whose value types diverge from the declared schema
Show propertiesHide properties
TestRunSchemaTypeMismatchfieldstringrequiredexpectedstringrequiredDeclared schema type, e.g. STRING / DOUBLE
actualstringrequiredInferred type of the value the LLM returned
anysystemPromptstring | anyComposed system prompt the LLM actually received - base template + auto-injected schema fields + tool descriptions + customInstructions. Identical to what Flink sends at deploy time (built by the same build_system_prompt). Null when the run errored before reaching the LLM (e.g. llm_resolution_error).
Show propertiesHide properties
stringanyuserMessagestring | anyFirst user message dispatched to the LLM, after the prompts.user template wraps the (input-fields-filtered) sample record. Lets the test pane show the exact pair of strings the LLM saw on iteration 1. Null when the run errored before message construction.
Show propertiesHide properties
stringanyerrorTestRunError | anyShow propertiesHide properties
kindstringrequiredOne of: llm_resolution_error, llm_provider_error, timeout, unsupported_provider, http_connection_resolution_error, …
messagestringrequiredanyValidation Error
detailValidationError[]Show propertiesHide properties
ValidationErrorlocstring | integer[]requiredShow propertiesHide properties
string | integerstringintegermsgstringrequiredtypestringrequiredinputanyctxobject