flemming-it
305ab1cecd
New optional input api: ollama|openai|anthropic (default ollama — existing flows unchanged). - openai: OpenAI Chat Completions format (/v1/chat/completions), Bearer auth — targets vLLM and compatible self-hosted servers. - anthropic: Messages API (/v1/messages), x-api-key + anthropic-version headers, top-level system field, mandatory max_tokens (fixed 4096). - Audit outputs unchanged: model_endpoint/model_name always set; model_digest stays Ollama-only (no digest API on openai/anthropic, field is left empty rather than fabricated). - api_key is only ever placed in auth headers; never in outputs, errors, or audit events (verified against the event log). - No streaming, no tool calls. Bump module + capability version to 0.2.0. Wire-format unit tests for request serialization and response parsing against fixed JSON fixtures; live smoke green on the ollama path and on the openai path against an OpenAI-compatible local endpoint. Signed-off-by: flemming-it <sf@flemming.it>
64 lines
2.2 KiB
Markdown
64 lines
2.2 KiB
Markdown
# llm-chat
|
|
|
|
F∆I module providing the `llm.chat` capability — a generic
|
|
LLM chat adapter speaking Ollama (default), OpenAI Chat
|
|
Completions (OpenAI, vLLM, compatible servers), and the
|
|
Anthropic Messages API, selected via the optional `api` input.
|
|
|
|
## Capability
|
|
|
|
| Field | Value |
|
|
|-------|-------|
|
|
| Capability | `llm.chat@0.2.0` |
|
|
| Inputs | `prompt: text`, `endpoint: text`, `model: text`, `api_key: text` (opt), `system_prompt: text` (opt), `api: text` (opt: `ollama`\|`openai`\|`anthropic`) |
|
|
| Outputs | `response: text`, `model_endpoint: text`, `model_name: text`, `model_digest: text` |
|
|
| Permissions | `net: localhost`, `net: 127.0.0.1`, `net: api.openai.com`, `net: api.anthropic.com` |
|
|
| Status (in store index) | `alpha` |
|
|
|
|
## Why a separate module from orchestrator-llm
|
|
|
|
`orchestrator-llm` wraps an LLM call inside a planning prompt
|
|
that emits a structured F∆I `Plan`. `llm.chat` is the lower-
|
|
level adapter: it sends a prompt and returns the assistant text
|
|
verbatim. Flows that need a generic chat step (summarisation,
|
|
translation, free-form Q&A) compose `llm.chat` directly; flows
|
|
that need plan generation use `orchestrator-llm`.
|
|
|
|
## Audit fields
|
|
|
|
`model_endpoint`, `model_name`, and `model_digest` are emitted
|
|
on every successful invocation. Together they answer the audit
|
|
question "which exact model produced this response?" The digest
|
|
probe targets Ollama's `/api/show` and is best-effort — non-
|
|
Ollama endpoints and transient probe failures yield an empty
|
|
digest rather than blocking the response. Cf. Ch∆In
|
|
`docs/advanced/compliance-gaps.md` Gap 1.
|
|
|
|
## Build
|
|
|
|
```bash
|
|
cargo build --release --target wasm32-wasip2
|
|
```
|
|
|
|
## Test
|
|
|
|
```bash
|
|
cargo test # wire-format unit tests, all host-side
|
|
cargo build --release --target wasm32-wasip2
|
|
```
|
|
|
|
## SDK source
|
|
|
|
The Cargo.toml git-deps `chain-module-sdk` from
|
|
`https://git.flemming.ws/fai/module-sdk.git`. The Forgejo
|
|
instance has `REQUIRE_SIGNIN_VIEW=true`, so anonymous clones
|
|
fail; CI uses the `MODULE_SDK_PAT` actions secret + `git config
|
|
url.X.insteadOf Y` to authenticate cargo's git fetch
|
|
transparently. Local dev uses the same pattern via env vars.
|
|
|
|
## License
|
|
|
|
Apache-2.0.
|
|
|
|
Author: Dr. Stefan Flemming, Flemming.AI <chain@flemming.ai>
|
|
Repository: https://git.flemming.ws/chain-modules/llm-chat
|