feat: report what a call cost in tokens
Three dialects spell the same two numbers differently: Ollama counts in `prompt_eval_count` and `eval_count`, the OpenAI shape in `usage.prompt_tokens` and `usage.completion_tokens`, Anthropic in `usage.input_tokens` and `usage.output_tokens`. All three now reach the caller as `tokens_prompt` and `tokens_completion`, which the hub lifts into the audit log by itself. An answer that carries no count leaves both fields empty rather than reporting a zero: a zero is a measurement, and this is the absence of one. Half a count is reported as far as it goes. The cost of a model call was the one part of the audit question "which model produced this, and what did it cost" that nothing answered (model pool M3). Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
parent
a6fe962916
commit
d5b000e18e
8 changed files with 155 additions and 7 deletions
2
Cargo.lock
generated
2
Cargo.lock
generated
|
|
@ -138,7 +138,7 @@ checksum = "6cc46bac87ef8093eed6f272babb833b6443374399985ac8ed28471ee0918545"
|
|||
|
||||
[[package]]
|
||||
name = "llm_chat"
|
||||
version = "0.2.1"
|
||||
version = "0.3.0"
|
||||
dependencies = [
|
||||
"chain-module-sdk",
|
||||
"serde",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue