docs: add MODULE.md + MODULE.de.md inline docs
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 1s
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 1s
Explains the 'why this module instead of inline HTTP' answer operators ask when they see llm.chat for the first time: audit, permission posture, endpoint portability. Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
parent
4c10a5ff3d
commit
84a23e6b14
2 changed files with 210 additions and 0 deletions
106
MODULE.de.md
Normal file
106
MODULE.de.md
Normal file
|
|
@ -0,0 +1,106 @@
|
||||||
|
# llm.chat
|
||||||
|
|
||||||
|
Generischer Ollama-kompatibler LLM-Chat-Adapter. Das
|
||||||
|
Baustein-Modul für jeden Flow, der eine LLM-Einzelantwort
|
||||||
|
braucht — klassifizieren, Felder extrahieren, umformulieren,
|
||||||
|
entscheiden — ohne einen eigenen HTTP-Client zu bauen.
|
||||||
|
|
||||||
|
## Capability
|
||||||
|
|
||||||
|
- `llm.chat@0.1.0`
|
||||||
|
|
||||||
|
## Eingaben
|
||||||
|
|
||||||
|
| Name | Typ | Beschreibung |
|
||||||
|
| ---------------- | ---- | ------------------------------------------------------------------------- |
|
||||||
|
| `prompt` | text | Der User-Prompt. |
|
||||||
|
| `endpoint` | text | Ollama-förmiger `/api/chat`-URL (z.B. `http://localhost:11434/api/chat`). |
|
||||||
|
| `model` | text | Modell-ID (z.B. `qwen2.5:14b`, `llama3.1:8b`). |
|
||||||
|
| `api_key` | text | Optionaler Bearer-Token für Cloud-Endpunkte. |
|
||||||
|
| `system_prompt` | text | Optionale System-Nachricht. Leer = Default des Modells. |
|
||||||
|
|
||||||
|
## Ausgaben
|
||||||
|
|
||||||
|
| Name | Typ | Beschreibung |
|
||||||
|
| ---------------- | ---- | --------------------------------------------------------------------- |
|
||||||
|
| `response` | text | Die Antwort des Assistants als Plain-Text. |
|
||||||
|
| `model_endpoint` | text | URL, gegen die die Antwort erzeugt wurde. |
|
||||||
|
| `model_name` | text | Modell-ID wie an die LLM-API gesendet. |
|
||||||
|
| `model_digest` | text | SHA-256-Digest des bedienenden Ollama-Modells. Leer bei Cloud-APIs. |
|
||||||
|
|
||||||
|
Zusammen beantworten die drei `model_*`-Ausgaben die Audit-
|
||||||
|
Frage: „Welches genaue Modell hat diese Antwort erzeugt?" —
|
||||||
|
diese Spur ist der Grund, warum Flows dieses Modul nutzen statt
|
||||||
|
eigene HTTP-Calls zu sprechen.
|
||||||
|
|
||||||
|
## Berechtigungen
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
permissions:
|
||||||
|
- "net: localhost"
|
||||||
|
- "net: 127.0.0.1"
|
||||||
|
- "net: api.openai.com"
|
||||||
|
- "net: api.anthropic.com"
|
||||||
|
```
|
||||||
|
|
||||||
|
Loopback (lokales Ollama) per Default. Cloud-Endpunkte brauchen
|
||||||
|
einen Operator-Policy-Override in
|
||||||
|
`~/.fai/config.yaml#security.max_permissions`.
|
||||||
|
|
||||||
|
## Warum dieses Modul statt Inline-HTTP
|
||||||
|
|
||||||
|
Drei Gründe:
|
||||||
|
|
||||||
|
1. **Audit.** Jeder LLM-Aufruf erzeugt `model_endpoint`,
|
||||||
|
`model_name`, `model_digest` als separate Ausgaben, die
|
||||||
|
im hash-verketteten Audit-Log neben der Response landen.
|
||||||
|
Eine Aufsichts-Reproduzierbarkeits-Prüfung vergleicht
|
||||||
|
einfach das Digest-Feld mit dem Modell-Snapshot.
|
||||||
|
2. **Permission-Posture.** Operator:innen sehen `llm.chat`
|
||||||
|
in den installierten Modulen + dessen deklarierte
|
||||||
|
`net:`-Liste. Ein hausgemachtes HTTP-Modul würde seine
|
||||||
|
Endpunkte entweder verstecken oder bei jeder Installation
|
||||||
|
eine frische Review-Oberfläche schaffen.
|
||||||
|
3. **Endpunkt-Portabilität.** Endpunkt + Modell sind Flow-
|
||||||
|
Inputs, keine Compile-Time-Konstanten. Derselbe Flow läuft
|
||||||
|
in der Entwicklung gegen `localhost:11434` und in Produktion
|
||||||
|
gegen ein Inferenz-Cluster — nur die Eingabe wechselt.
|
||||||
|
|
||||||
|
## Grenzen in v0.1.0
|
||||||
|
|
||||||
|
- Kein Streaming. Die ganze Antwort wird gepuffert, bevor der
|
||||||
|
Output-Step feuert.
|
||||||
|
- Keine Tool-Call- / Function-Call-Oberfläche. Ein Flow, der
|
||||||
|
Tool-Use braucht, kombiniert mehrere `llm.chat`-Schritte mit
|
||||||
|
Prompt-Engineering oder nutzt MCP via Bridge.
|
||||||
|
- Cloud-Provider-Adapter für OpenAI und Anthropic kommen erst,
|
||||||
|
wenn ein Flow sie braucht. Heute ist das Ollama-Wire-Format
|
||||||
|
das einzige Ziel.
|
||||||
|
|
||||||
|
## Beispiel-Flow
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: classify-incoming
|
||||||
|
inputs:
|
||||||
|
text: text
|
||||||
|
steps:
|
||||||
|
- id: classify
|
||||||
|
use: llm.chat@^0
|
||||||
|
with:
|
||||||
|
prompt: $inputs.text
|
||||||
|
system_prompt: |
|
||||||
|
Klassifiziere den Text als: question, complaint,
|
||||||
|
feedback, spam. Antworte nur mit dem Label.
|
||||||
|
endpoint: "http://localhost:11434/api/chat"
|
||||||
|
model: "qwen2.5:14b"
|
||||||
|
outputs:
|
||||||
|
category: $classify.response
|
||||||
|
audit_model: $classify.model_digest
|
||||||
|
```
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo build --release --target wasm32-wasip2
|
||||||
|
# Ausgabe: target/wasm32-wasip2/release/llm_chat.wasm
|
||||||
|
```
|
||||||
104
MODULE.md
Normal file
104
MODULE.md
Normal file
|
|
@ -0,0 +1,104 @@
|
||||||
|
# llm.chat
|
||||||
|
|
||||||
|
Generic Ollama-compatible LLM chat adapter. The building-block
|
||||||
|
module any flow uses when it needs a one-shot LLM completion —
|
||||||
|
classify, extract-fields, rewrite, decide — instead of rolling
|
||||||
|
its own HTTP client.
|
||||||
|
|
||||||
|
## Capability
|
||||||
|
|
||||||
|
- `llm.chat@0.1.0`
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
| Name | Type | Description |
|
||||||
|
| ---------------- | ---- | --------------------------------------------------------------- |
|
||||||
|
| `prompt` | text | The user-facing prompt. |
|
||||||
|
| `endpoint` | text | Ollama-shaped `/api/chat` URL (e.g. `http://localhost:11434/api/chat`). |
|
||||||
|
| `model` | text | Model identifier (e.g. `qwen2.5:14b`, `llama3.1:8b`). |
|
||||||
|
| `api_key` | text | Optional bearer token for cloud-hosted endpoints. |
|
||||||
|
| `system_prompt` | text | Optional system message. Empty = use the model's default. |
|
||||||
|
|
||||||
|
## Outputs
|
||||||
|
|
||||||
|
| Name | Type | Description |
|
||||||
|
| ---------------- | ---- | ------------------------------------------------------------------- |
|
||||||
|
| `response` | text | The assistant's plain-text reply. |
|
||||||
|
| `model_endpoint` | text | The endpoint the response was generated against. |
|
||||||
|
| `model_name` | text | The model identifier as supplied to the LLM API. |
|
||||||
|
| `model_digest` | text | SHA-256 digest of the served Ollama model. Empty for cloud APIs. |
|
||||||
|
|
||||||
|
Together the three `model_*` outputs answer the audit
|
||||||
|
question: "which exact model produced this response?" — that
|
||||||
|
audit trail is the reason flows use this module rather than
|
||||||
|
spawning their own HTTP calls.
|
||||||
|
|
||||||
|
## Permissions
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
permissions:
|
||||||
|
- "net: localhost"
|
||||||
|
- "net: 127.0.0.1"
|
||||||
|
- "net: api.openai.com"
|
||||||
|
- "net: api.anthropic.com"
|
||||||
|
```
|
||||||
|
|
||||||
|
Loopback (local Ollama) by default. Cloud endpoints require an
|
||||||
|
operator-policy override in `~/.fai/config.yaml#security.max_permissions`.
|
||||||
|
|
||||||
|
## Why this module instead of inline HTTP
|
||||||
|
|
||||||
|
Three reasons:
|
||||||
|
|
||||||
|
1. **Audit.** Every LLM call surfaces `model_endpoint`,
|
||||||
|
`model_name`, `model_digest` as separate outputs that land
|
||||||
|
in the hash-chained audit log alongside the response. A
|
||||||
|
regulator-facing reproducibility check just compares the
|
||||||
|
digest field to the model snapshot.
|
||||||
|
2. **Permission posture.** The operator sees `llm.chat` in
|
||||||
|
the installed modules + its declared `net:` list. A
|
||||||
|
home-grown HTTP module would either hide its endpoints
|
||||||
|
or be a fresh review surface every time.
|
||||||
|
3. **Endpoint portability.** The endpoint + model are flow
|
||||||
|
inputs, not compile-time constants. The same flow runs
|
||||||
|
against `localhost:11434` in dev and a production
|
||||||
|
inference cluster in prod just by swapping the input.
|
||||||
|
|
||||||
|
## Limits in v0.1.0
|
||||||
|
|
||||||
|
- No streaming. The whole reply is buffered before the output
|
||||||
|
step fires.
|
||||||
|
- No tool-call / function-call surface. A flow needing tool
|
||||||
|
use composes multiple `llm.chat` steps with prompt
|
||||||
|
engineering, or uses MCP via the bridge.
|
||||||
|
- Cloud-provider adapters for OpenAI and Anthropic are
|
||||||
|
deferred until a flow actually needs them. Today the
|
||||||
|
Ollama wire-format is the only target.
|
||||||
|
|
||||||
|
## Example flow
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: classify-incoming
|
||||||
|
inputs:
|
||||||
|
text: text
|
||||||
|
steps:
|
||||||
|
- id: classify
|
||||||
|
use: llm.chat@^0
|
||||||
|
with:
|
||||||
|
prompt: $inputs.text
|
||||||
|
system_prompt: |
|
||||||
|
Classify the text as one of: question, complaint,
|
||||||
|
feedback, spam. Answer with the label only.
|
||||||
|
endpoint: "http://localhost:11434/api/chat"
|
||||||
|
model: "qwen2.5:14b"
|
||||||
|
outputs:
|
||||||
|
category: $classify.response
|
||||||
|
audit_model: $classify.model_digest
|
||||||
|
```
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo build --release --target wasm32-wasip2
|
||||||
|
# Output: target/wasm32-wasip2/release/llm_chat.wasm
|
||||||
|
```
|
||||||
Loading…
Add table
Add a link
Reference in a new issue