feat: report what a call cost in tokens
Three dialects spell the same two numbers differently: Ollama counts in `prompt_eval_count` and `eval_count`, the OpenAI shape in `usage.prompt_tokens` and `usage.completion_tokens`, Anthropic in `usage.input_tokens` and `usage.output_tokens`. All three now reach the caller as `tokens_prompt` and `tokens_completion`, which the hub lifts into the audit log by itself. An answer that carries no count leaves both fields empty rather than reporting a zero: a zero is a measurement, and this is the absence of one. Half a count is reported as far as it goes. The cost of a model call was the one part of the audit question "which model produced this, and what did it cost" that nothing answered (model pool M3). Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
parent
a6fe962916
commit
d5b000e18e
8 changed files with 155 additions and 7 deletions
|
|
@ -10,7 +10,7 @@ Anthropic Messages API.
|
|||
|
||||
## Capability
|
||||
|
||||
- `llm.chat@0.2.0`
|
||||
- `llm.chat@0.3.0`
|
||||
|
||||
## Inputs
|
||||
|
||||
|
|
@ -40,6 +40,8 @@ Anthropic Messages API.
|
|||
| `model_endpoint` | text | The endpoint the response was generated against. |
|
||||
| `model_name` | text | The model identifier as supplied to the LLM API. |
|
||||
| `model_digest` | text | SHA-256 digest of the served Ollama model. Empty for `openai`/`anthropic` — those APIs expose no digest; the field is left empty rather than fabricated. |
|
||||
| `tokens_prompt` | text | What the prompt cost, as the backend counted it. Empty when the answer carried no count. |
|
||||
| `tokens_completion` | text | What the answer cost, same rule. Ollama counts in `eval_count`, the OpenAI shape in `usage.completion_tokens`, Anthropic in `usage.output_tokens`. |
|
||||
|
||||
Together the three `model_*` outputs answer the audit
|
||||
question: "which exact model produced this response?" — that
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue