feat: OpenAI and Anthropic wire-format adapters via new api input
New optional input api: ollama|openai|anthropic (default ollama — existing flows unchanged). - openai: OpenAI Chat Completions format (/v1/chat/completions), Bearer auth — targets vLLM and compatible self-hosted servers. - anthropic: Messages API (/v1/messages), x-api-key + anthropic-version headers, top-level system field, mandatory max_tokens (fixed 4096). - Audit outputs unchanged: model_endpoint/model_name always set; model_digest stays Ollama-only (no digest API on openai/anthropic, field is left empty rather than fabricated). - api_key is only ever placed in auth headers; never in outputs, errors, or audit events (verified against the event log). - No streaming, no tool calls. Bump module + capability version to 0.2.0. Wire-format unit tests for request serialization and response parsing against fixed JSON fixtures; live smoke green on the ollama path and on the openai path against an OpenAI-compatible local endpoint. Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
parent
b5389fce45
commit
305ab1cecd
9 changed files with 662 additions and 136 deletions
76
MODULE.md
76
MODULE.md
|
|
@ -1,23 +1,35 @@
|
|||
# llm.chat
|
||||
|
||||
Generic Ollama-compatible LLM chat adapter. The building-block
|
||||
module any flow uses when it needs a one-shot LLM completion —
|
||||
classify, extract-fields, rewrite, decide — instead of rolling
|
||||
its own HTTP client.
|
||||
Generic LLM chat adapter. The building-block module any flow
|
||||
uses when it needs a one-shot LLM completion — classify,
|
||||
extract-fields, rewrite, decide — instead of rolling its own
|
||||
HTTP client. Speaks three wire formats, selected by the
|
||||
optional `api` input: Ollama (default), OpenAI Chat
|
||||
Completions (OpenAI, vLLM, compatible servers), and the
|
||||
Anthropic Messages API.
|
||||
|
||||
## Capability
|
||||
|
||||
- `llm.chat@0.1.0`
|
||||
- `llm.chat@0.2.0`
|
||||
|
||||
## Inputs
|
||||
|
||||
| Name | Type | Description |
|
||||
| ---------------- | ---- | --------------------------------------------------------------- |
|
||||
| `prompt` | text | The user-facing prompt. |
|
||||
| `endpoint` | text | Ollama-shaped `/api/chat` URL (e.g. `http://localhost:11434/api/chat`). |
|
||||
| `model` | text | Model identifier (e.g. `qwen2.5:14b`, `llama3.1:8b`). |
|
||||
| `api_key` | text | Optional bearer token for cloud-hosted endpoints. |
|
||||
| `endpoint` | text | Chat endpoint URL matching the selected `api` (see below). |
|
||||
| `model` | text | Model identifier (e.g. `qwen2.5:14b`, `claude-fable-5`). |
|
||||
| `api_key` | text | Optional API key. Never appears in outputs, logs, or events. |
|
||||
| `system_prompt` | text | Optional system message. Empty = use the model's default. |
|
||||
| `api` | text | Optional wire format: `ollama` (default), `openai`, `anthropic`. |
|
||||
|
||||
### Wire formats (`api`)
|
||||
|
||||
| `api` | Endpoint shape | Auth header | Notes |
|
||||
| ----------- | --------------------------------------------- | ---------------------------------- | ------------------------------------------------------------ |
|
||||
| `ollama` | `http://localhost:11434/api/chat` | `Authorization: Bearer` (optional) | v0.1.0 behavior, unchanged. Model-digest probe via `/api/show`. |
|
||||
| `openai` | `https://.../v1/chat/completions` | `Authorization: Bearer` | OpenAI Chat Completions format — also spoken by vLLM and most self-hosted inference servers. |
|
||||
| `anthropic` | `https://api.anthropic.com/v1/messages` | `x-api-key` + `anthropic-version` | Messages API: `system` is a top-level field; `max_tokens` is mandatory and defaults to 4096. |
|
||||
|
||||
## Outputs
|
||||
|
||||
|
|
@ -26,7 +38,7 @@ its own HTTP client.
|
|||
| `response` | text | The assistant's plain-text reply. |
|
||||
| `model_endpoint` | text | The endpoint the response was generated against. |
|
||||
| `model_name` | text | The model identifier as supplied to the LLM API. |
|
||||
| `model_digest` | text | SHA-256 digest of the served Ollama model. Empty for cloud APIs. |
|
||||
| `model_digest` | text | SHA-256 digest of the served Ollama model. Empty for `openai`/`anthropic` — those APIs expose no digest; the field is left empty rather than fabricated. |
|
||||
|
||||
Together the three `model_*` outputs answer the audit
|
||||
question: "which exact model produced this response?" — that
|
||||
|
|
@ -43,7 +55,7 @@ permissions:
|
|||
- "net: api.anthropic.com"
|
||||
```
|
||||
|
||||
Loopback (local Ollama) by default. Cloud endpoints require an
|
||||
Loopback (local Ollama / vLLM) by default. Cloud endpoints require an
|
||||
operator-policy override in `~/.chain/config.yaml#security.max_permissions`.
|
||||
|
||||
## Why this module instead of inline HTTP
|
||||
|
|
@ -59,23 +71,22 @@ Three reasons:
|
|||
the installed modules + its declared `net:` list. A
|
||||
home-grown HTTP module would either hide its endpoints
|
||||
or be a fresh review surface every time.
|
||||
3. **Endpoint portability.** The endpoint + model are flow
|
||||
inputs, not compile-time constants. The same flow runs
|
||||
against `localhost:11434` in dev and a production
|
||||
inference cluster in prod just by swapping the input.
|
||||
3. **Endpoint portability.** The endpoint, model, and wire
|
||||
format are flow inputs, not compile-time constants. The
|
||||
same flow runs against `localhost:11434` in dev and a
|
||||
production vLLM cluster in prod just by swapping inputs.
|
||||
|
||||
## Limits in v0.1.0
|
||||
## Limits in v0.2.0
|
||||
|
||||
- No streaming. The whole reply is buffered before the output
|
||||
step fires.
|
||||
- No tool-call / function-call surface. A flow needing tool
|
||||
use composes multiple `llm.chat` steps with prompt
|
||||
engineering, or uses MCP via the bridge.
|
||||
- Cloud-provider adapters for OpenAI and Anthropic are
|
||||
deferred until a flow actually needs them. Today the
|
||||
Ollama wire-format is the only target.
|
||||
- Anthropic `max_tokens` is fixed at 4096 (the API requires
|
||||
the field; a configurable input lands when a flow needs it).
|
||||
|
||||
## Example flow
|
||||
## Example flows
|
||||
|
||||
```yaml
|
||||
name: classify-incoming
|
||||
|
|
@ -96,6 +107,33 @@ outputs:
|
|||
audit_model: $classify.model_digest
|
||||
```
|
||||
|
||||
Against a vLLM (OpenAI-compatible) server:
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- id: classify
|
||||
use: llm.chat@^0
|
||||
with:
|
||||
api: "openai"
|
||||
endpoint: "http://inference.internal:8000/v1/chat/completions"
|
||||
model: "meta-llama/Llama-3.1-8B-Instruct"
|
||||
prompt: $inputs.text
|
||||
```
|
||||
|
||||
Against the Anthropic Messages API:
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- id: classify
|
||||
use: llm.chat@^0
|
||||
with:
|
||||
api: "anthropic"
|
||||
endpoint: "https://api.anthropic.com/v1/messages"
|
||||
model: "claude-fable-5"
|
||||
api_key: $inputs.anthropic_key
|
||||
prompt: $inputs.text
|
||||
```
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue