feat: report what a call cost in tokens
Three dialects spell the same two numbers differently: Ollama counts in `prompt_eval_count` and `eval_count`, the OpenAI shape in `usage.prompt_tokens` and `usage.completion_tokens`, Anthropic in `usage.input_tokens` and `usage.output_tokens`. All three now reach the caller as `tokens_prompt` and `tokens_completion`, which the hub lifts into the audit log by itself. An answer that carries no count leaves both fields empty rather than reporting a zero: a zero is a measurement, and this is the absence of one. Half a count is reported as far as it goes. The cost of a model call was the one part of the audit question "which model produced this, and what did it cost" that nothing answered (model pool M3). Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
parent
a6fe962916
commit
d5b000e18e
8 changed files with 155 additions and 7 deletions
30
module.yaml
30
module.yaml
|
|
@ -1,12 +1,12 @@
|
|||
schema_version: 3
|
||||
provider: chain
|
||||
name: llm-chat
|
||||
version: 0.2.2
|
||||
version: 0.3.0
|
||||
|
||||
# Capability provided by this module.
|
||||
provides:
|
||||
- capability: llm.chat
|
||||
version: 0.2.2
|
||||
version: 0.3.0
|
||||
|
||||
# Declared inputs in verbose v3 form. Studio renders the
|
||||
# description in the active locale as a tooltip on each
|
||||
|
|
@ -171,6 +171,32 @@ outputs:
|
|||
keine Digest-API bereit). Zusammen mit model_endpoint und
|
||||
model_name beantwortet das die Audit-Frage "welches genaue
|
||||
Modell hat diese Antwort erzeugt?"
|
||||
tokens_prompt:
|
||||
type: text
|
||||
description:
|
||||
en: |
|
||||
How many tokens the prompt cost, as the backend counted it.
|
||||
Empty when the answer carried no count: a zero would read
|
||||
like a measurement nobody took. The hub lifts this field
|
||||
into the audit log automatically.
|
||||
de: |
|
||||
Wie viele Token die Anfrage gekostet hat, nach der Zählung
|
||||
des Backends. Leer, wenn die Antwort keine Zählung enthielt:
|
||||
eine Null würde wie eine Messung aussehen, die niemand
|
||||
gemacht hat. Der Hub hebt dieses Feld selbst in den Prüfpfad.
|
||||
tokens_completion:
|
||||
type: text
|
||||
description:
|
||||
en: |
|
||||
How many tokens the answer cost, same rules as
|
||||
tokens_prompt. Ollama counts in `eval_count`, the OpenAI
|
||||
shape in `usage.completion_tokens`, Anthropic in
|
||||
`usage.output_tokens`.
|
||||
de: |
|
||||
Wie viele Token die Antwort gekostet hat, gleiche Regeln wie
|
||||
tokens_prompt. Ollama zählt in `eval_count`, das
|
||||
OpenAI-Format in `usage.completion_tokens`, Anthropic in
|
||||
`usage.output_tokens`.
|
||||
|
||||
# Permissions required.
|
||||
#
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue