llm-chat/MODULE.md

llm.chat

Generic Ollama-compatible LLM chat adapter. The building-block module any flow uses when it needs a one-shot LLM completion — classify, extract-fields, rewrite, decide — instead of rolling its own HTTP client.

Capability

• llm.chat@0.1.0

Inputs

Name	Type	Description
prompt	text	The user-facing prompt.
endpoint	text	Ollama-shaped /api/chat URL (e.g. http://localhost:11434/api/chat).
model	text	Model identifier (e.g. qwen2.5:14b, llama3.1:8b).
api_key	text	Optional bearer token for cloud-hosted endpoints.
system_prompt	text	Optional system message. Empty = use the model's default.

Outputs

Name	Type	Description
response	text	The assistant's plain-text reply.
model_endpoint	text	The endpoint the response was generated against.
model_name	text	The model identifier as supplied to the LLM API.
model_digest	text	SHA-256 digest of the served Ollama model. Empty for cloud APIs.

Together the three model_* outputs answer the audit question: "which exact model produced this response?" — that audit trail is the reason flows use this module rather than spawning their own HTTP calls.

Permissions

permissions:
  - "net: localhost"
  - "net: 127.0.0.1"
  - "net: api.openai.com"
  - "net: api.anthropic.com"

Loopback (local Ollama) by default. Cloud endpoints require an operator-policy override in ~/.chain/config.yaml#security.max_permissions.

Why this module instead of inline HTTP

Three reasons:

1. Audit. Every LLM call surfaces model_endpoint, model_name, model_digest as separate outputs that land in the hash-chained audit log alongside the response. A regulator-facing reproducibility check just compares the digest field to the model snapshot.
2. Permission posture. The operator sees llm.chat in the installed modules + its declared net: list. A home-grown HTTP module would either hide its endpoints or be a fresh review surface every time.
3. Endpoint portability. The endpoint + model are flow inputs, not compile-time constants. The same flow runs against localhost:11434 in dev and a production inference cluster in prod just by swapping the input.

Limits in v0.1.0

• No streaming. The whole reply is buffered before the output step fires.
• No tool-call / function-call surface. A flow needing tool use composes multiple llm.chat steps with prompt engineering, or uses MCP via the bridge.
• Cloud-provider adapters for OpenAI and Anthropic are deferred until a flow actually needs them. Today the Ollama wire-format is the only target.

Example flow

name: classify-incoming
inputs:
  text: text
steps:
  - id: classify
    use: llm.chat@^0
    with:
      prompt: $inputs.text
      system_prompt: |
        Classify the text as one of: question, complaint,
        feedback, spam. Answer with the label only.
      endpoint: "http://localhost:11434/api/chat"
      model: "qwen2.5:14b"
outputs:
  category: $classify.response
  audit_model: $classify.model_digest

Build

cargo build --release --target wasm32-wasip2
# Output: target/wasm32-wasip2/release/llm_chat.wasm