feat: sampling options (temperature, top_p, top_k, seed)
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
aila asked for these twice (07.08. and 08.08.) and worked around it by baking the values into a derived Ollama model, which is the only way they could get a reproducible pipeline. Verified against a live Ollama: two runs with `temperature: "0"` and `seed: "42"` return the same answer. Each wire format gets what it actually supports, and an option the wire does not have is REFUSED rather than dropped: `top_k` on the OpenAI wire, `seed` on Anthropic. A silently ignored setting produces a run that looks configured and is not, which is worse than an error, especially for `seed`, where the whole point is being able to repeat the run. A non-numeric value is an error too, so a typo cannot pass as "server default". Unset leaves the request byte-identical to 0.2.1, so no existing caller sees a change. Eight guards covering all three wires, both refusals, the unset case, and that `seed` stays a whole number. Capability 0.2.1 to 0.2.2: four optional inputs, nothing removed. Signed-off-by: Stefan Flemming <stefan.a.flemming@googlemail.com>
This commit is contained in:
parent
9483f196a1
commit
a6fe962916
4 changed files with 315 additions and 21 deletions
45
module.yaml
45
module.yaml
|
|
@ -1,12 +1,12 @@
|
|||
schema_version: 3
|
||||
provider: chain
|
||||
name: llm-chat
|
||||
version: 0.2.1
|
||||
version: 0.2.2
|
||||
|
||||
# Capability provided by this module.
|
||||
provides:
|
||||
- capability: llm.chat
|
||||
version: 0.2.1
|
||||
version: 0.2.2
|
||||
|
||||
# Declared inputs in verbose v3 form. Studio renders the
|
||||
# description in the active locale as a tooltip on each
|
||||
|
|
@ -67,6 +67,47 @@ inputs:
|
|||
"Authorization: Bearer", bei anthropic als "x-api-key"
|
||||
gesendet. Leer = kein Auth-Header. Erscheint nie in
|
||||
Ausgaben, Logs oder Audit-Events.
|
||||
temperature:
|
||||
type: text
|
||||
description:
|
||||
en: |
|
||||
Sampling temperature, e.g. "0" for the most repeatable
|
||||
output. Empty leaves the server default. Works on all three
|
||||
wire formats.
|
||||
de: |
|
||||
Sampling-Temperatur, z. B. "0" für möglichst wiederholbare
|
||||
Ausgaben. Leer lässt die Server-Voreinstellung. Gilt für
|
||||
alle drei Wire-Formate.
|
||||
top_p:
|
||||
type: text
|
||||
description:
|
||||
en: Nucleus sampling cutoff. Empty leaves the server default.
|
||||
de: Nucleus-Sampling-Grenze. Leer lässt die Server-Voreinstellung.
|
||||
top_k:
|
||||
type: text
|
||||
description:
|
||||
en: |
|
||||
Top-k cutoff, a whole number. Ollama and Anthropic only; with
|
||||
api openai or vllm the input is refused rather than ignored,
|
||||
because a setting that silently does nothing produces a run
|
||||
that looks configured and is not.
|
||||
de: |
|
||||
Top-k-Grenze, ganze Zahl. Nur Ollama und Anthropic; mit api
|
||||
openai oder vllm wird der Eingang abgelehnt statt ignoriert,
|
||||
weil eine still wirkungslose Einstellung einen Lauf erzeugt,
|
||||
der konfiguriert aussieht und es nicht ist.
|
||||
seed:
|
||||
type: text
|
||||
description:
|
||||
en: |
|
||||
Sampling seed, a whole number. Together with temperature 0
|
||||
this is what makes a run repeatable. Ollama and the OpenAI
|
||||
wire only; Anthropic has no seed and refuses the input.
|
||||
de: |
|
||||
Sampling-Seed, ganze Zahl. Zusammen mit Temperatur 0 macht
|
||||
das einen Lauf wiederholbar. Nur Ollama und das
|
||||
OpenAI-Format; Anthropic kennt keinen Seed und lehnt den
|
||||
Eingang ab.
|
||||
keep_alive:
|
||||
type: text
|
||||
description:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue