feat: sampling options (temperature, top_p, top_k, seed)
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s

aila asked for these twice (07.08. and 08.08.) and worked around it
by baking the values into a derived Ollama model, which is the only
way they could get a reproducible pipeline.

Verified against a live Ollama: two runs with `temperature: "0"` and
`seed: "42"` return the same answer.

Each wire format gets what it actually supports, and an option the
wire does not have is REFUSED rather than dropped: `top_k` on the
OpenAI wire, `seed` on Anthropic. A silently ignored setting
produces a run that looks configured and is not, which is worse than
an error, especially for `seed`, where the whole point is being able
to repeat the run. A non-numeric value is an error too, so a typo
cannot pass as "server default".

Unset leaves the request byte-identical to 0.2.1, so no existing
caller sees a change.

Eight guards covering all three wires, both refusals, the
unset case, and that `seed` stays a whole number.

Capability 0.2.1 to 0.2.2: four optional inputs, nothing removed.

Signed-off-by: Stefan Flemming <stefan.a.flemming@googlemail.com>
This commit is contained in:
Stefan Flemming 2026-10-16 20:47:40 +02:00
parent 9483f196a1
commit a6fe962916
4 changed files with 315 additions and 21 deletions

View file

@ -1,12 +1,12 @@
schema_version: 3
provider: chain
name: llm-chat
version: 0.2.1
version: 0.2.2
# Capability provided by this module.
provides:
- capability: llm.chat
version: 0.2.1
version: 0.2.2
# Declared inputs in verbose v3 form. Studio renders the
# description in the active locale as a tooltip on each
@ -67,6 +67,47 @@ inputs:
"Authorization: Bearer", bei anthropic als "x-api-key"
gesendet. Leer = kein Auth-Header. Erscheint nie in
Ausgaben, Logs oder Audit-Events.
temperature:
type: text
description:
en: |
Sampling temperature, e.g. "0" for the most repeatable
output. Empty leaves the server default. Works on all three
wire formats.
de: |
Sampling-Temperatur, z. B. "0" für möglichst wiederholbare
Ausgaben. Leer lässt die Server-Voreinstellung. Gilt für
alle drei Wire-Formate.
top_p:
type: text
description:
en: Nucleus sampling cutoff. Empty leaves the server default.
de: Nucleus-Sampling-Grenze. Leer lässt die Server-Voreinstellung.
top_k:
type: text
description:
en: |
Top-k cutoff, a whole number. Ollama and Anthropic only; with
api openai or vllm the input is refused rather than ignored,
because a setting that silently does nothing produces a run
that looks configured and is not.
de: |
Top-k-Grenze, ganze Zahl. Nur Ollama und Anthropic; mit api
openai oder vllm wird der Eingang abgelehnt statt ignoriert,
weil eine still wirkungslose Einstellung einen Lauf erzeugt,
der konfiguriert aussieht und es nicht ist.
seed:
type: text
description:
en: |
Sampling seed, a whole number. Together with temperature 0
this is what makes a run repeatable. Ollama and the OpenAI
wire only; Anthropic has no seed and refuses the input.
de: |
Sampling-Seed, ganze Zahl. Zusammen mit Temperatur 0 macht
das einen Lauf wiederholbar. Nur Ollama und das
OpenAI-Format; Anthropic kennt keinen Seed und lehnt den
Eingang ab.
keep_alive:
type: text
description: