Three dialects spell the same two numbers differently: Ollama counts in `prompt_eval_count` and `eval_count`, the OpenAI shape in `usage.prompt_tokens` and `usage.completion_tokens`, Anthropic in `usage.input_tokens` and `usage.output_tokens`. All three now reach the caller as `tokens_prompt` and `tokens_completion`, which the hub lifts into the audit log by itself. An answer that carries no count leaves both fields empty rather than reporting a zero: a zero is a measurement, and this is the absence of one. Half a count is reported as far as it goes. The cost of a model call was the one part of the audit question "which model produced this, and what did it cost" that nothing answered (model pool M3). Signed-off-by: flemming-it <sf@flemming.it>
212 lines
8.2 KiB
YAML
212 lines
8.2 KiB
YAML
schema_version: 3
|
|
provider: chain
|
|
name: llm-chat
|
|
version: 0.3.0
|
|
|
|
# Capability provided by this module.
|
|
provides:
|
|
- capability: llm.chat
|
|
version: 0.3.0
|
|
|
|
# Declared inputs in verbose v3 form. Studio renders the
|
|
# description in the active locale as a tooltip on each
|
|
# port; the editor falls back to the English description
|
|
# when no peer is configured.
|
|
inputs:
|
|
prompt:
|
|
type: text
|
|
description:
|
|
en: The user-facing prompt text the model should respond to.
|
|
de: Der vom Modell zu beantwortende Prompt-Text.
|
|
endpoint:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Chat endpoint URL matching the selected api:
|
|
ollama "http://localhost:11434/api/chat",
|
|
openai "https://.../v1/chat/completions" (vLLM etc.),
|
|
anthropic "https://api.anthropic.com/v1/messages".
|
|
de: |
|
|
Chat-Endpunkt-URL passend zur gewählten api:
|
|
ollama "http://localhost:11434/api/chat",
|
|
openai "https://.../v1/chat/completions" (vLLM u. a.),
|
|
anthropic "https://api.anthropic.com/v1/messages".
|
|
api:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Optional wire format: "ollama" (default), "vllm"
|
|
(self-hosted vLLM), "openai" (OpenAI or other
|
|
OpenAI-compatible servers such as LM Studio), or
|
|
"anthropic" (Messages API). vllm and openai speak the
|
|
same wire; the separate value exists so you can write
|
|
what you mean and get vLLM-specific hints. Empty = ollama.
|
|
de: |
|
|
Optionales Wire-Format: "ollama" (Default), "vllm"
|
|
(selbst gehostetes vLLM), "openai" (OpenAI oder andere
|
|
OpenAI-kompatible Server wie LM Studio) oder "anthropic"
|
|
(Messages API). vllm und openai sprechen dieselbe Wire;
|
|
der eigene Wert existiert, damit Du schreibst, was Du
|
|
meinst, und vLLM-spezifische Hinweise bekommst.
|
|
Leer = ollama.
|
|
model:
|
|
type: text
|
|
description:
|
|
en: Model identifier as registered with the endpoint, e.g. "qwen2.5:14b" or "llama3.1:8b".
|
|
de: Modell-Identifier wie beim Endpunkt registriert, z. B. "qwen2.5:14b" oder "llama3.1:8b".
|
|
api_key:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Optional API key. Sent as "Authorization: Bearer" for
|
|
ollama/openai and as "x-api-key" for anthropic. Empty
|
|
string means no auth header is sent. Never appears in
|
|
outputs, logs, or audit events.
|
|
de: |
|
|
Optionaler API-Key. Bei ollama/openai als
|
|
"Authorization: Bearer", bei anthropic als "x-api-key"
|
|
gesendet. Leer = kein Auth-Header. Erscheint nie in
|
|
Ausgaben, Logs oder Audit-Events.
|
|
temperature:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Sampling temperature, e.g. "0" for the most repeatable
|
|
output. Empty leaves the server default. Works on all three
|
|
wire formats.
|
|
de: |
|
|
Sampling-Temperatur, z. B. "0" für möglichst wiederholbare
|
|
Ausgaben. Leer lässt die Server-Voreinstellung. Gilt für
|
|
alle drei Wire-Formate.
|
|
top_p:
|
|
type: text
|
|
description:
|
|
en: Nucleus sampling cutoff. Empty leaves the server default.
|
|
de: Nucleus-Sampling-Grenze. Leer lässt die Server-Voreinstellung.
|
|
top_k:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Top-k cutoff, a whole number. Ollama and Anthropic only; with
|
|
api openai or vllm the input is refused rather than ignored,
|
|
because a setting that silently does nothing produces a run
|
|
that looks configured and is not.
|
|
de: |
|
|
Top-k-Grenze, ganze Zahl. Nur Ollama und Anthropic; mit api
|
|
openai oder vllm wird der Eingang abgelehnt statt ignoriert,
|
|
weil eine still wirkungslose Einstellung einen Lauf erzeugt,
|
|
der konfiguriert aussieht und es nicht ist.
|
|
seed:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Sampling seed, a whole number. Together with temperature 0
|
|
this is what makes a run repeatable. Ollama and the OpenAI
|
|
wire only; Anthropic has no seed and refuses the input.
|
|
de: |
|
|
Sampling-Seed, ganze Zahl. Zusammen mit Temperatur 0 macht
|
|
das einen Lauf wiederholbar. Nur Ollama und das
|
|
OpenAI-Format; Anthropic kennt keinen Seed und lehnt den
|
|
Eingang ab.
|
|
keep_alive:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Ollama only: how long the server keeps the model resident
|
|
after this request, e.g. "4h", "30m", or "-1" for
|
|
indefinitely. Empty leaves the server default, which unloads
|
|
after five minutes, so the next first request pays a full
|
|
model reload (30 to 40 seconds for a large local model).
|
|
Setting this with any other api is refused rather than
|
|
ignored, because the input expresses an expectation the
|
|
other backends cannot meet.
|
|
de: |
|
|
Nur für Ollama: wie lange der Server das Modell nach dieser
|
|
Anfrage geladen lässt, z. B. "4h", "30m" oder "-1" für
|
|
unbegrenzt. Leer bedeutet Server-Voreinstellung, die nach
|
|
fünf Minuten entlädt; die nächste erste Anfrage zahlt dann
|
|
den vollen Modell-Neustart (30 bis 40 Sekunden bei einem
|
|
großen lokalen Modell). Mit einer anderen api wird der Wert
|
|
abgelehnt statt ignoriert, weil er eine Erwartung ausdrückt,
|
|
die die anderen Backends nicht erfüllen können.
|
|
system_prompt:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Optional system prompt. Empty string omits the system
|
|
message and lets the model use its built-in default.
|
|
de: |
|
|
Optionaler System-Prompt. Leer = das Modell verwendet
|
|
seinen eingebauten Default.
|
|
|
|
# Outputs produced.
|
|
outputs:
|
|
response:
|
|
type: text
|
|
description:
|
|
en: The assistant's plain-text reply.
|
|
de: Die Klartext-Antwort des Modells.
|
|
model_endpoint:
|
|
type: text
|
|
description:
|
|
en: The endpoint the response was generated against. Used for audit correlation.
|
|
de: Der Endpunkt, gegen den die Antwort erzeugt wurde — für Audit-Korrelation.
|
|
model_name:
|
|
type: text
|
|
description:
|
|
en: The model identifier as supplied to the LLM API.
|
|
de: Der an die LLM-API übergebene Modell-Identifier.
|
|
model_digest:
|
|
type: text
|
|
description:
|
|
en: |
|
|
SHA-256 digest of the served Ollama model, when reachable.
|
|
Empty for non-Ollama endpoints (OpenAI / Anthropic do not
|
|
expose a digest API). Together with model_endpoint and
|
|
model_name this answers the audit question "which exact
|
|
model produced this response?"
|
|
de: |
|
|
SHA-256-Digest des bedienten Ollama-Modells, wenn erreichbar.
|
|
Leer für Nicht-Ollama-Endpunkte (OpenAI / Anthropic stellen
|
|
keine Digest-API bereit). Zusammen mit model_endpoint und
|
|
model_name beantwortet das die Audit-Frage "welches genaue
|
|
Modell hat diese Antwort erzeugt?"
|
|
tokens_prompt:
|
|
type: text
|
|
description:
|
|
en: |
|
|
How many tokens the prompt cost, as the backend counted it.
|
|
Empty when the answer carried no count: a zero would read
|
|
like a measurement nobody took. The hub lifts this field
|
|
into the audit log automatically.
|
|
de: |
|
|
Wie viele Token die Anfrage gekostet hat, nach der Zählung
|
|
des Backends. Leer, wenn die Antwort keine Zählung enthielt:
|
|
eine Null würde wie eine Messung aussehen, die niemand
|
|
gemacht hat. Der Hub hebt dieses Feld selbst in den Prüfpfad.
|
|
tokens_completion:
|
|
type: text
|
|
description:
|
|
en: |
|
|
How many tokens the answer cost, same rules as
|
|
tokens_prompt. Ollama counts in `eval_count`, the OpenAI
|
|
shape in `usage.completion_tokens`, Anthropic in
|
|
`usage.output_tokens`.
|
|
de: |
|
|
Wie viele Token die Antwort gekostet hat, gleiche Regeln wie
|
|
tokens_prompt. Ollama zählt in `eval_count`, das
|
|
OpenAI-Format in `usage.completion_tokens`, Anthropic in
|
|
`usage.output_tokens`.
|
|
|
|
# Permissions required.
|
|
#
|
|
# The module makes outbound HTTP to the configured endpoint.
|
|
# Default declarations cover loopback (Ollama) and the common
|
|
# cloud providers. Operators with different endpoints fork
|
|
# module.yaml in Phase 0.5; operator-config-driven permission
|
|
# overrides arrive in Phase 1+.
|
|
permissions:
|
|
- "net: localhost"
|
|
- "net: 127.0.0.1"
|
|
- "net: api.openai.com"
|
|
- "net: api.anthropic.com"
|