feat: max_tokens and repeat_penalty inputs, truncated and stop_reason outputs, 0.4.0
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 1s
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 1s
A model caught in a repetition loop wrote until the flow's 300 s timeout. max_tokens bounds the answer on every wire (Ollama num_predict, OpenAI max_tokens, Anthropic max_tokens), and truncated tells the caller whether the answer stopped at that limit. repeat_penalty goes to Ollama and vLLM and is refused where the backend has no such setting. Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
parent
14c63083d7
commit
72f3037ebe
8 changed files with 361 additions and 9 deletions
2
Cargo.lock
generated
2
Cargo.lock
generated
|
|
@ -138,7 +138,7 @@ checksum = "6cc46bac87ef8093eed6f272babb833b6443374399985ac8ed28471ee0918545"
|
|||
|
||||
[[package]]
|
||||
name = "llm_chat"
|
||||
version = "0.3.1"
|
||||
version = "0.4.0"
|
||||
dependencies = [
|
||||
"chain-module-sdk",
|
||||
"serde",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue