llm-chat/MODULE.md

# llm.chat

Generic LLM chat adapter. The building-block module any flow
uses when it needs a one-shot LLM completion — classify,
extract-fields, rewrite, decide — instead of rolling its own
HTTP client. Speaks three wire formats, selected by the
optional `api` input: Ollama (default), OpenAI Chat
Completions (OpenAI, vLLM, compatible servers), and the
Anthropic Messages API.

## Capability

- `llm.chat@0.2.0`

## Inputs

| Name             | Type | Description                                                     |
| ---------------- | ---- | --------------------------------------------------------------- |
| `prompt`         | text | The user-facing prompt.                                         |
| `endpoint`       | text | Chat endpoint URL matching the selected `api` (see below).      |
| `model`          | text | Model identifier (e.g. `qwen2.5:14b`, `claude-fable-5`).      |
| `api_key`        | text | Optional API key. Never appears in outputs, logs, or events.    |
| `system_prompt`  | text | Optional system message. Empty = use the model's default.       |
| `api`            | text | Optional wire format: `ollama` (default), `openai`, `anthropic`. |

### Wire formats (`api`)

| `api`       | Endpoint shape                                | Auth header                        | Notes                                                        |
| ----------- | --------------------------------------------- | ---------------------------------- | ------------------------------------------------------------ |
| `ollama`    | `http://localhost:11434/api/chat`             | `Authorization: Bearer` (optional) | v0.1.0 behavior, unchanged. Model-digest probe via `/api/show`. |
| `openai`    | `https://.../v1/chat/completions`             | `Authorization: Bearer`            | OpenAI Chat Completions format — also spoken by vLLM and most self-hosted inference servers. |
| `anthropic` | `https://api.anthropic.com/v1/messages`       | `x-api-key` + `anthropic-version`  | Messages API: `system` is a top-level field; `max_tokens` is mandatory and defaults to 4096. |

## Outputs

| Name             | Type | Description                                                         |
| ---------------- | ---- | ------------------------------------------------------------------- |
| `response`       | text | The assistant's plain-text reply.                                   |
| `model_endpoint` | text | The endpoint the response was generated against.                    |
| `model_name`     | text | The model identifier as supplied to the LLM API.                    |
| `model_digest`   | text | SHA-256 digest of the served Ollama model. Empty for `openai`/`anthropic` — those APIs expose no digest; the field is left empty rather than fabricated. |

Together the three `model_*` outputs answer the audit
question: "which exact model produced this response?" — that
audit trail is the reason flows use this module rather than
spawning their own HTTP calls.

## Permissions

```yaml
permissions:
  - "net: localhost"
  - "net: 127.0.0.1"
  - "net: api.openai.com"
  - "net: api.anthropic.com"
```

Loopback (local Ollama / vLLM) by default. Cloud endpoints require an
operator-policy override in `~/.chain/config.yaml#security.max_permissions`.

## Why this module instead of inline HTTP

Three reasons:

1. **Audit.** Every LLM call surfaces `model_endpoint`,
   `model_name`, `model_digest` as separate outputs that land
   in the hash-chained audit log alongside the response. A
   regulator-facing reproducibility check just compares the
   digest field to the model snapshot.
2. **Permission posture.** The operator sees `llm.chat` in
   the installed modules + its declared `net:` list. A
   home-grown HTTP module would either hide its endpoints
   or be a fresh review surface every time.
3. **Endpoint portability.** The endpoint, model, and wire
   format are flow inputs, not compile-time constants. The
   same flow runs against `localhost:11434` in dev and a
   production vLLM cluster in prod just by swapping inputs.

## Limits in v0.2.0

- No streaming. The whole reply is buffered before the output
  step fires.
- No tool-call / function-call surface. A flow needing tool
  use composes multiple `llm.chat` steps with prompt
  engineering, or uses MCP via the bridge.
- Anthropic `max_tokens` is fixed at 4096 (the API requires
  the field; a configurable input lands when a flow needs it).

## Example flows

```yaml
name: classify-incoming
inputs:
  text: text
steps:
  - id: classify
    use: llm.chat@^0
    with:
      prompt: $inputs.text
      system_prompt: |
        Classify the text as one of: question, complaint,
        feedback, spam. Answer with the label only.
      endpoint: "http://localhost:11434/api/chat"
      model: "qwen2.5:14b"
outputs:
  category: $classify.response
  audit_model: $classify.model_digest
```

Against a vLLM (OpenAI-compatible) server:

```yaml
steps:
  - id: classify
    use: llm.chat@^0
    with:
      api: "openai"
      endpoint: "http://inference.internal:8000/v1/chat/completions"
      model: "meta-llama/Llama-3.1-8B-Instruct"
      prompt: $inputs.text
```

Against the Anthropic Messages API:

```yaml
steps:
  - id: classify
    use: llm.chat@^0
    with:
      api: "anthropic"
      endpoint: "https://api.anthropic.com/v1/messages"
      model: "claude-fable-5"
      api_key: $inputs.anthropic_key
      prompt: $inputs.text
```

## Build

```bash
cargo build --release --target wasm32-wasip2
# Output: target/wasm32-wasip2/release/llm_chat.wasm
```