chain-modules/text-translate
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions

feat: selectable wire format — ollama (default), openai (vLLM etc.), anthropic
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
Details
sign-bundle / sign (push) Failing after 1m51s
Details

Browse source 
Port the multi-API client from llm.chat (llm.rs kept in lockstep):
the optional api input selects the dialect, default stays the
unchanged v0.1.x Ollama behavior. openai covers vLLM, LM Studio,
LiteLLM and cloud OpenAI; api-specific error hints; api_key sent
as Bearer (ollama/openai) or x-api-key (anthropic). model_digest
stays an Ollama-only best-effort probe and is documented as such.

Proven end-to-end against a hermetic OpenAI-wire fake (request
shape validated, response parsed) via a hub flow run.

Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
flemming-it 2026-08-20 16:32:59 +02:00
parent 47db969564
commit c54c14b28a
6 changed files with 567 additions and 93 deletions
Download patch file Download diff file Expand all files Collapse all files

2
Cargo.toml
View file

@ -5,7 +5,7 @@
[package]
name = "text_translate"
version = "0.1.0"
version = "0.2.0"
edition = "2024"
license = "Apache-2.0"
publish = false

7
MODULE.de.md
View file

@ -1,6 +1,6 @@
# text.translate
Ollama-basierte Übersetzung zwischen Sprachen. Sendet Quelltext
LLM-basierte Übersetzung zwischen Sprachen (Ollama, vLLM, jeder OpenAI-kompatible Server oder Anthropic). Sendet Quelltext
an einen konfigurierten LLM-Endpunkt mit einem
Übersetzungs-System-Prompt und liefert die Übersetzung plus
audit-taugliche Modell-Herkunfts-Felder.
@ -16,7 +16,8 @@ audit-taugliche Modell-Herkunfts-Felder.
| `text` | text | Quelltext. |
| `target_language` | text | Zielsprache in einfachem Englisch (z.B. `German`, `French`, `ja-JP`). |
| `source_language` | text | Optionaler Hinweis auf die Ausgangssprache. Leer = Modell erkennt selbst. |
| `endpoint` | text | Ollama-förmiger `/api/chat`-Endpunkt. |
| `endpoint` | text | Chat-Endpunkt passend zum gewählten `api` (Ollama `/api/chat`, OpenAI-kompatibel `/v1/chat/completions` — vLLM u. a., Anthropic `/v1/messages`). |
| `api` | text | Optionales Wire-Format: `ollama` (Default), `openai`, `anthropic`. |
| `model` | text | Modell-ID am Endpunkt (z.B. `qwen2.5:14b`). |
| `api_key` | text | Optionaler Bearer-Token für Cloud-Endpunkte. |
@ -29,7 +30,7 @@ audit-taugliche Modell-Herkunfts-Felder.
| `target_language` | text | Echo der Eingabe. |
| `model_endpoint` | text | URL, gegen die übersetzt wurde. |
| `model_name` | text | Modell-ID wie an die LLM-API gesendet. |
| `model_digest` | text | SHA-256-Digest des bedienenden Ollama-Modells (wenn erreichbar). |
| `model_digest` | text | SHA-256-Digest des bedienten Modells — nur bei Ollama, sonst leer. |
Die drei `model_*`-Ausgaben sind das Audit-Primitiv: jede
Übersetzung im Flow-Log lässt sich auch Monate später dem exakten

7
MODULE.md
View file

@ -1,6 +1,6 @@
# text.translate
Ollama-backed translation between languages. Sends source text
LLM-backed translation between languages (Ollama, vLLM, any OpenAI-compatible server, or Anthropic). Sends source text
to a configured LLM endpoint with a translation system prompt
and emits the translated text plus audit-grade
model-provenance fields.
@ -16,7 +16,8 @@ model-provenance fields.
| `text` | text | Source text to translate. |
| `target_language` | text | Target language in plain English (e.g. `German`, `French`, `ja-JP`). |
| `source_language` | text | Optional source-language hint. Empty = let the model auto-detect. |
| `endpoint` | text | Ollama-shaped `/api/chat` endpoint URL. |
| `endpoint` | text | Chat endpoint URL matching the selected `api` (Ollama `/api/chat`, OpenAI-compatible `/v1/chat/completions` — vLLM etc., Anthropic `/v1/messages`). |
| `api` | text | Optional wire format: `ollama` (default), `openai`, `anthropic`. |
| `model` | text | Model identifier the endpoint serves (e.g. `qwen2.5:14b`). |
| `api_key` | text | Optional bearer token for cloud-hosted endpoints. |
@ -29,7 +30,7 @@ model-provenance fields.
| `target_language` | text | Echo of the input value. |
| `model_endpoint` | text | URL the translation was generated against. |
| `model_name` | text | Model identifier as supplied to the LLM API. |
| `model_digest` | text | SHA-256 digest of the served Ollama model (when reachable). |
| `model_digest` | text | SHA-256 digest of the served model — Ollama only, empty elsewhere. |
The three `model_*` outputs are the audit primitive: every
translation in the flow log can be matched to the exact model

51
module.yaml
View file

@ -1,11 +1,11 @@
schema_version: 3
provider: chain
name: text-translate
version: 0.1.0
version: 0.2.0
provides:
- capability: text.translate
version: 0.1.0
version: 0.2.0
inputs:
text:
@ -35,18 +35,43 @@ inputs:
endpoint:
type: text
description:
en: Ollama-shaped /api/chat endpoint.
de: Ollama-kompatibler /api/chat-Endpunkt.
en: |
Chat endpoint URL matching the selected api:
ollama "http://localhost:11434/api/chat",
openai "http://localhost:8000/v1/chat/completions"
(vLLM etc.), anthropic "https://.../v1/messages".
de: |
Chat-Endpunkt-URL passend zum gewählten api:
ollama "http://localhost:11434/api/chat",
openai "http://localhost:8000/v1/chat/completions"
(vLLM u. a.), anthropic "https://.../v1/messages".
api:
type: text
description:
en: |
Optional wire format: "ollama" (default), "openai"
(OpenAI-compatible servers such as vLLM or LM Studio),
or "anthropic" (Messages API). Empty = ollama.
de: |
Optionales Wire-Format: "ollama" (Default), "openai"
(OpenAI-kompatible Server wie vLLM oder LM Studio)
oder "anthropic" (Messages API). Leer = ollama.
model:
type: text
description:
en: Model identifier (e.g. "qwen2.5:14b").
de: Modell-Identifier (z. B. "qwen2.5:14b").
en: Model identifier as registered with the endpoint (e.g. "qwen2.5:14b").
de: Modell-Identifier wie am Endpunkt registriert (z. B. "qwen2.5:14b").
api_key:
type: text
description:
en: Optional bearer token for cloud-hosted endpoints.
de: Optionaler Bearer-Token für Cloud-Endpunkte.
en: |
Optional API key. Sent as "Authorization: Bearer" for
ollama/openai and as "x-api-key" for anthropic. Empty
for local endpoints.
de: |
Optionaler API-Key. Bei ollama/openai als
"Authorization: Bearer", bei anthropic als "x-api-key"
gesendet. Leer für lokale Endpunkte.
outputs:
translation:
@ -77,8 +102,14 @@ outputs:
model_digest:
type: text
description:
en: SHA-256 digest of the served Ollama model (when reachable).
de: SHA-256-Digest des bedienten Ollama-Modells (wenn erreichbar).
en: |
SHA-256 digest of the served model, best-effort probe.
Ollama only — empty for other endpoints (OpenAI / vLLM /
Anthropic expose no digest API).
de: |
SHA-256-Digest des bedienten Modells (Best-Effort-Probe).
Nur bei Ollama — leer bei anderen Endpunkten (OpenAI /
vLLM / Anthropic bieten keine Digest-API).
permissions:
- "net: localhost"

82
src/lib.rs
View file

@ -1,12 +1,15 @@
//! `text.translate` — Ollama-backed text translation.
//! `text.translate` — LLM-backed text translation.
//!
//! Sends `text` plus a target-language directive to an Ollama
//! `/api/chat` endpoint and returns the translated text. Reports
//! Sends `text` plus a target-language directive to an LLM chat
//! endpoint and returns the translated text. Reports
//! `model_endpoint`, `model_name`, and `model_digest` as audit
//! fields, same as `llm.chat`.
//!
//! v0.1.0 targets Ollama only. Cloud-provider adapters (OpenAI,
//! Anthropic) follow when a flow needs them.
//! The wire format is selected by the optional `api` input:
//! `ollama` (default, unchanged v0.1.x behavior), `openai`
//! (OpenAI-compatible `/v1/chat/completions` — vLLM, LM Studio,
//! LiteLLM, cloud OpenAI), or `anthropic` (Messages API). The
//! client logic in `llm.rs` is kept in lockstep with `llm.chat`.
mod llm;
@ -32,11 +35,15 @@ pub fn invoke(_ctx: Context, inputs: Inputs) -> Result<Outputs, ModuleError> {
.get("source_language")
.and_then(payload_text)
.unwrap_or_default();
let api_raw = inputs.get("api").and_then(payload_text).unwrap_or_default();
let api =
crate::llm::Api::parse(&api_raw).map_err(|e| ModuleError::invalid_input(e.to_string()))?;
let prompt = build_prompt(&source_language, &target_language, &text);
let client = make_client();
let params = crate::llm::ChatParams {
api,
endpoint: &endpoint,
model: &model,
api_key: &api_key,
@ -44,7 +51,7 @@ pub fn invoke(_ctx: Context, inputs: Inputs) -> Result<Outputs, ModuleError> {
prompt: &prompt,
};
let result = crate::llm::chat_with_identity(&client, &params)
.map_err(|e| llm_error_to_module_error(e, &endpoint, &model))?;
.map_err(|e| llm_error_to_module_error(e, api, &endpoint, &model))?;
Ok(Outputs::new()
.with_text("translation", result.response)
@ -57,27 +64,51 @@ pub fn invoke(_ctx: Context, inputs: Inputs) -> Result<Outputs, ModuleError> {
/// Turn a transport/protocol error into a message that names the
/// likely cause and the fix, instead of a raw `ConnectionRefused`.
fn llm_error_to_module_error(e: crate::llm::LlmError, endpoint: &str, model: &str) -> ModuleError {
use crate::llm::LlmError;
match e {
LlmError::Http(detail) => ModuleError::internal(format!(
/// The hints are api-specific: an Ollama connect failure almost
/// always means Ollama isn't running or the model isn't pulled,
/// while cloud/vLLM failures are usually endpoint or key issues.
/// The message must never contain the api_key.
fn llm_error_to_module_error(
e: crate::llm::LlmError,
api: crate::llm::Api,
endpoint: &str,
model: &str,
) -> ModuleError {
use crate::llm::{Api, LlmError};
match (api, e) {
(Api::Ollama, LlmError::Http(detail)) => ModuleError::internal(format!(
"LLM endpoint {endpoint} not reachable ({detail}). Is Ollama running? \
Start it with `ollama serve`, then pull the model with `ollama pull {model}`. \
If the LLM runs elsewhere, set the `endpoint` input to its /api URL."
)),
LlmError::Status(404) => ModuleError::internal(format!(
(Api::Openai, LlmError::Http(detail)) => ModuleError::internal(format!(
"LLM endpoint {endpoint} not reachable ({detail}). Expected an \
OpenAI-compatible server (OpenAI, vLLM, ...) at a /v1/chat/completions URL."
)),
(Api::Anthropic, LlmError::Http(detail)) => ModuleError::internal(format!(
"LLM endpoint {endpoint} not reachable ({detail}). Expected the \
Anthropic Messages API at a /v1/messages URL."
)),
(Api::Ollama, LlmError::Status(404)) => ModuleError::internal(format!(
"LLM endpoint {endpoint} returned 404 for model '{model}' — the model is \
likely not pulled. Run `ollama pull {model}` (or check the model name)."
)),
LlmError::Status(code) => ModuleError::internal(format!(
(_, LlmError::Status(401)) | (_, LlmError::Status(403)) => ModuleError::internal(format!(
"LLM endpoint {endpoint} rejected the request as unauthorized — check the \
`api_key` input."
)),
(_, LlmError::Status(code)) => ModuleError::internal(format!(
"LLM endpoint {endpoint} returned HTTP {code} for model '{model}'."
)),
LlmError::Decode(detail) => ModuleError::internal(format!(
"LLM response from {endpoint} was not valid Ollama JSON: {detail}"
(_, LlmError::Decode(detail)) => ModuleError::internal(format!(
"LLM response from {endpoint} did not match the expected schema: {detail}"
)),
LlmError::MissingInput(name) => {
(_, LlmError::MissingInput(name)) => {
ModuleError::invalid_input(format!("missing required input '{name}'"))
}
(_, LlmError::UnsupportedApi(raw)) => ModuleError::invalid_input(format!(
"unsupported api '{raw}' (expected: ollama, openai, anthropic)"
)),
}
}
@ -120,7 +151,7 @@ impl crate::llm::LlmClient for HostStubClient {
&self,
_url: &str,
_body: &str,
_api_key: &str,
_headers: &[(&'static str, String)],
) -> Result<String, crate::llm::LlmError> {
Err(crate::llm::LlmError::Http(
"LLM HTTP path is unavailable on the host build; only wasm32 supports outbound HTTP"
@ -138,14 +169,14 @@ impl crate::llm::LlmClient for WakiClient {
&self,
url: &str,
body: &str,
api_key: &str,
headers: &[(&'static str, String)],
) -> Result<String, crate::llm::LlmError> {
let mut request = waki::Client::new()
.post(url)
.header("Content-Type", "application/json")
.body(body.to_string());
if !api_key.is_empty() {
request = request.header("Authorization", &format!("Bearer {api_key}"));
for (name, value) in headers {
request = request.header(*name, value);
}
let response = request
.send()
@ -179,4 +210,17 @@ mod tests {
let p = build_prompt("English", "French", "Hello");
assert!(p.contains("English text into French"));
}
#[test]
fn empty_api_input_defaults_to_ollama() {
assert_eq!(crate::llm::Api::parse("").unwrap(), crate::llm::Api::Ollama);
}
#[test]
fn openai_api_is_accepted_for_vllm_endpoints() {
assert_eq!(
crate::llm::Api::parse("openai").unwrap(),
crate::llm::Api::Openai
);
}
}

511
src/llm.rs
View file

@ -1,35 +1,84 @@
//! Ollama-shaped chat client.
//! Multi-API chat client: Ollama (default), OpenAI, Anthropic.
//!
//! v0.1.0 targets the Ollama `/api/chat` endpoint. OpenAI- and
//! Anthropic-compatible adapters are deliberately deferred — each
//! has its own request/response shape that warrants its own crate
//! (or at least its own module here) once a flow needs it.
//! The wire format is selected via the optional `api` input:
//!
//! - `ollama` (default) — Ollama `/api/chat`; unchanged v0.1.0
//! behavior including the best-effort model-digest probe.
//! - `openai` — OpenAI Chat Completions wire format
//! (`/v1/chat/completions`), the de-facto standard implemented
//! by vLLM and most self-hosted inference servers. Bearer auth.
//! - `anthropic` — Anthropic Messages API (`/v1/messages`),
//! `x-api-key` + `anthropic-version` headers, top-level
//! `system` field, mandatory `max_tokens`.
//!
//! No streaming, no tool calls — one prompt in, one completion out.
//!
//! All HTTP I/O lives behind a `LlmClient` trait so unit tests can
//! exercise prompt building, response parsing, and the digest
//! exercise request building, response parsing, and the digest
//! probe on the host without making real network calls.
use serde::Serialize;
/// Anthropic's Messages API requires `max_tokens`. This default is
/// large enough for document-processing completions while staying
/// well below every current model's output cap.
const ANTHROPIC_DEFAULT_MAX_TOKENS: u32 = 4096;
/// Pinned Messages API version header. Anthropic keeps old
/// versions working; bump deliberately, never implicitly.
const ANTHROPIC_VERSION: &str = "2023-06-01";
#[allow(dead_code)]
#[derive(Debug, thiserror::Error)]
pub enum LlmError {
#[error("missing required input '{0}'")]
MissingInput(&'static str),
#[error("unsupported api '{0}' (expected: ollama, openai, anthropic)")]
UnsupportedApi(String),
#[error("http error: {0}")]
Http(String),
#[error("non-success status: {0}")]
Status(u16),
#[error("response body could not be parsed as Ollama schema: {0}")]
#[error("response body could not be parsed as the expected schema: {0}")]
Decode(String),
}
/// Which wire format to speak. Selected by the `api` input.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Api {
Ollama,
Openai,
Anthropic,
}
impl Api {
/// Parse the `api` input. Empty string means the input was not
/// provided and falls back to Ollama (v0.1.0 behavior).
pub fn parse(raw: &str) -> Result<Self, LlmError> {
match raw.trim().to_ascii_lowercase().as_str() {
"" | "ollama" => Ok(Api::Ollama),
"openai" => Ok(Api::Openai),
"anthropic" => Ok(Api::Anthropic),
other => Err(LlmError::UnsupportedApi(other.to_string())),
}
}
}
pub trait LlmClient {
fn post_json(&self, url: &str, body: &str, api_key: &str) -> Result<String, LlmError>;
/// POST `body` as JSON to `url` with the given extra headers.
/// `Content-Type: application/json` is implied. Header VALUES
/// may carry credentials — implementations must never log them.
fn post_json(
&self,
url: &str,
body: &str,
headers: &[(&'static str, String)],
) -> Result<String, LlmError>;
}
#[derive(Debug, Clone)]
pub struct ChatParams<'a> {
pub api: Api,
pub endpoint: &'a str,
pub model: &'a str,
pub api_key: &'a str,
@ -37,8 +86,29 @@ pub struct ChatParams<'a> {
pub prompt: &'a str,
}
/// Auth/protocol headers for the selected API. The api_key is only
/// ever placed into a header value here; it must not appear in any
/// error, output, or log line.
pub fn build_headers(p: &ChatParams) -> Vec<(&'static str, String)> {
let mut headers = Vec::with_capacity(2);
match p.api {
Api::Ollama | Api::Openai => {
if !p.api_key.is_empty() {
headers.push(("Authorization", format!("Bearer {}", p.api_key)));
}
}
Api::Anthropic => {
if !p.api_key.is_empty() {
headers.push(("x-api-key", p.api_key.to_string()));
}
headers.push(("anthropic-version", ANTHROPIC_VERSION.to_string()));
}
}
headers
}
#[derive(Serialize)]
struct OllamaMessage<'a> {
struct ChatMessage<'a> {
role: &'a str,
content: &'a str,
}
@ -46,33 +116,86 @@ struct OllamaMessage<'a> {
#[derive(Serialize)]
struct OllamaRequest<'a> {
model: &'a str,
messages: Vec<OllamaMessage<'a>>,
messages: Vec<ChatMessage<'a>>,
stream: bool,
}
/// Build the request body for an Ollama `/api/chat` invocation.
/// The system prompt is omitted from the messages list when empty
/// so a deployment can use the model's built-in system prompt.
pub fn build_ollama_body(p: &ChatParams) -> String {
#[derive(Serialize)]
struct OpenAiRequest<'a> {
model: &'a str,
messages: Vec<ChatMessage<'a>>,
stream: bool,
}
#[derive(Serialize)]
struct AnthropicRequest<'a> {
model: &'a str,
max_tokens: u32,
#[serde(skip_serializing_if = "Option::is_none")]
system: Option<&'a str>,
messages: Vec<ChatMessage<'a>>,
}
/// System-then-user message list shared by the Ollama and OpenAI
/// wire formats. The system message is omitted when empty so a
/// deployment can use the model's built-in system prompt.
fn build_messages<'a>(p: &ChatParams<'a>) -> Vec<ChatMessage<'a>> {
let mut messages = Vec::with_capacity(2);
if !p.system_prompt.is_empty() {
messages.push(OllamaMessage {
messages.push(ChatMessage {
role: "system",
content: p.system_prompt,
});
}
messages.push(OllamaMessage {
messages.push(ChatMessage {
role: "user",
content: p.prompt,
});
messages
}
/// Build the request body for an Ollama `/api/chat` invocation.
pub fn build_ollama_body(p: &ChatParams) -> String {
let req = OllamaRequest {
model: p.model,
messages,
messages: build_messages(p),
stream: false,
};
serde_json::to_string(&req).unwrap_or_else(|_| String::from("{}"))
}
/// Build the request body for an OpenAI-compatible
/// `/v1/chat/completions` invocation (OpenAI, vLLM, most
/// self-hosted inference servers).
pub fn build_openai_body(p: &ChatParams) -> String {
let req = OpenAiRequest {
model: p.model,
messages: build_messages(p),
stream: false,
};
serde_json::to_string(&req).unwrap_or_else(|_| String::from("{}"))
}
/// Build the request body for an Anthropic `/v1/messages`
/// invocation. `system` is a top-level field (not a message);
/// `max_tokens` is mandatory in the Messages API.
pub fn build_anthropic_body(p: &ChatParams) -> String {
let req = AnthropicRequest {
model: p.model,
max_tokens: ANTHROPIC_DEFAULT_MAX_TOKENS,
system: if p.system_prompt.is_empty() {
None
} else {
Some(p.system_prompt)
},
messages: vec![ChatMessage {
role: "user",
content: p.prompt,
}],
};
serde_json::to_string(&req).unwrap_or_else(|_| String::from("{}"))
}
/// Extract the assistant's message text from an Ollama
/// /api/chat response.
pub fn extract_ollama_content(body: &str) -> Result<String, LlmError> {
@ -85,16 +208,44 @@ pub fn extract_ollama_content(body: &str) -> Result<String, LlmError> {
.ok_or_else(|| LlmError::Decode("missing message.content".into()))
}
/// Extract the assistant text from an OpenAI Chat Completions
/// response: `choices[0].message.content`.
pub fn extract_openai_content(body: &str) -> Result<String, LlmError> {
let v: serde_json::Value =
serde_json::from_str(body).map_err(|e| LlmError::Decode(e.to_string()))?;
v.get("choices")
.and_then(|c| c.get(0))
.and_then(|c| c.get("message"))
.and_then(|m| m.get("content"))
.and_then(|c| c.as_str())
.map(|s| s.to_string())
.ok_or_else(|| LlmError::Decode("missing choices[0].message.content".into()))
}
/// Extract the assistant text from an Anthropic Messages
/// response: `content[0].text`.
pub fn extract_anthropic_content(body: &str) -> Result<String, LlmError> {
let v: serde_json::Value =
serde_json::from_str(body).map_err(|e| LlmError::Decode(e.to_string()))?;
v.get("content")
.and_then(|c| c.get(0))
.and_then(|c| c.get("text"))
.and_then(|t| t.as_str())
.map(|s| s.to_string())
.ok_or_else(|| LlmError::Decode("missing content[0].text".into()))
}
#[derive(Debug, Clone)]
pub struct ChatWithIdentity {
pub response: String,
pub model_digest: Option<String>,
}
/// Run an Ollama chat call AND probe the model digest. Probe is
/// best-effort — non-Ollama endpoints (no `/api/chat` suffix) and
/// transient failures yield `model_digest = None` rather than
/// failing the whole call.
/// Run one chat call in the selected wire format AND, for Ollama,
/// probe the model digest. The probe is best-effort — non-Ollama
/// APIs expose no digest endpoint, so `model_digest = None` there
/// (documented empty, never invented); transient probe failures on
/// Ollama also yield `None` rather than failing the whole call.
pub fn chat_with_identity<C: LlmClient>(
client: &C,
p: &ChatParams,
@ -108,10 +259,22 @@ pub fn chat_with_identity<C: LlmClient>(
if p.prompt.is_empty() {
return Err(LlmError::MissingInput("prompt"));
}
let body = build_ollama_body(p);
let response_body = client.post_json(p.endpoint, &body, p.api_key)?;
let response = extract_ollama_content(&response_body)?;
let model_digest = probe_model_digest(client, p);
let body = match p.api {
Api::Ollama => build_ollama_body(p),
Api::Openai => build_openai_body(p),
Api::Anthropic => build_anthropic_body(p),
};
let headers = build_headers(p);
let response_body = client.post_json(p.endpoint, &body, &headers)?;
let response = match p.api {
Api::Ollama => extract_ollama_content(&response_body)?,
Api::Openai => extract_openai_content(&response_body)?,
Api::Anthropic => extract_anthropic_content(&response_body)?,
};
let model_digest = match p.api {
Api::Ollama => probe_model_digest(client, p),
Api::Openai | Api::Anthropic => None,
};
Ok(ChatWithIdentity {
response,
model_digest,
@ -121,7 +284,8 @@ pub fn chat_with_identity<C: LlmClient>(
fn probe_model_digest<C: LlmClient>(client: &C, p: &ChatParams) -> Option<String> {
let show_url = derive_show_url(p.endpoint)?;
let body = serde_json::to_string(&serde_json::json!({ "name": p.model })).ok()?;
let response = client.post_json(&show_url, &body, p.api_key).ok()?;
let headers = build_headers(p);
let response = client.post_json(&show_url, &body, &headers).ok()?;
extract_show_digest(&response)
}
@ -160,20 +324,39 @@ mod tests {
use super::*;
use std::cell::RefCell;
/// (url, body, headers) of one recorded call.
type RecordedCall = (String, String, Vec<(String, String)>);
struct MockClient {
responses: RefCell<Vec<Result<String, LlmError>>>,
/// Every call, in call order.
calls: RefCell<Vec<RecordedCall>>,
}
impl MockClient {
fn new(responses: Vec<Result<String, LlmError>>) -> Self {
Self {
responses: RefCell::new(responses),
calls: RefCell::new(Vec::new()),
}
}
}
impl LlmClient for MockClient {
fn post_json(&self, _url: &str, _body: &str, _api_key: &str) -> Result<String, LlmError> {
fn post_json(
&self,
url: &str,
body: &str,
headers: &[(&'static str, String)],
) -> Result<String, LlmError> {
self.calls.borrow_mut().push((
url.to_string(),
body.to_string(),
headers
.iter()
.map(|(k, v)| (k.to_string(), v.clone()))
.collect(),
));
self.responses
.borrow_mut()
.pop()
@ -181,14 +364,42 @@ mod tests {
}
}
#[test]
fn ollama_body_includes_system_when_provided() {
let p = ChatParams {
fn params<'a>(api: Api) -> ChatParams<'a> {
ChatParams {
api,
endpoint: "http://x/api/chat",
model: "qwen",
api_key: "",
system_prompt: "be helpful",
system_prompt: "",
prompt: "hello",
}
}
// ---------------- api input parsing ----------------
#[test]
fn api_parse_defaults_to_ollama() {
assert_eq!(Api::parse("").unwrap(), Api::Ollama);
assert_eq!(Api::parse("ollama").unwrap(), Api::Ollama);
assert_eq!(Api::parse(" OpenAI ").unwrap(), Api::Openai);
assert_eq!(Api::parse("anthropic").unwrap(), Api::Anthropic);
}
#[test]
fn api_parse_rejects_unknown() {
assert!(matches!(
Api::parse("gemini"),
Err(LlmError::UnsupportedApi(_))
));
}
// ---------------- Ollama wire format (regression) ----------------
#[test]
fn ollama_body_includes_system_when_provided() {
let p = ChatParams {
system_prompt: "be helpful",
..params(Api::Ollama)
};
let body = build_ollama_body(&p);
let v: serde_json::Value = serde_json::from_str(&body).unwrap();
@ -198,18 +409,12 @@ mod tests {
assert_eq!(messages[0]["content"], "be helpful");
assert_eq!(messages[1]["role"], "user");
assert_eq!(messages[1]["content"], "hello");
assert_eq!(v["stream"], false);
}
#[test]
fn ollama_body_omits_system_when_empty() {
let p = ChatParams {
endpoint: "http://x/api/chat",
model: "qwen",
api_key: "",
system_prompt: "",
prompt: "hello",
};
let body = build_ollama_body(&p);
let body = build_ollama_body(&params(Api::Ollama));
let v: serde_json::Value = serde_json::from_str(&body).unwrap();
let messages = v["messages"].as_array().unwrap();
assert_eq!(messages.len(), 1);
@ -231,6 +436,208 @@ mod tests {
));
}
#[test]
fn ollama_bearer_header_only_with_key() {
assert!(build_headers(&params(Api::Ollama)).is_empty());
let p = ChatParams {
api_key: "sk-test",
..params(Api::Ollama)
};
assert_eq!(
build_headers(&p),
vec![("Authorization", "Bearer sk-test".to_string())]
);
}
// ---------------- OpenAI wire format ----------------
#[test]
fn openai_body_matches_chat_completions_format() {
let p = ChatParams {
api: Api::Openai,
endpoint: "http://vllm:8000/v1/chat/completions",
model: "meta-llama/Llama-3.1-8B-Instruct",
api_key: "sk-x",
system_prompt: "be terse",
prompt: "hello",
};
let body = build_openai_body(&p);
let v: serde_json::Value = serde_json::from_str(&body).unwrap();
assert_eq!(v["model"], "meta-llama/Llama-3.1-8B-Instruct");
assert_eq!(v["stream"], false);
let messages = v["messages"].as_array().unwrap();
assert_eq!(messages.len(), 2);
assert_eq!(messages[0]["role"], "system");
assert_eq!(messages[0]["content"], "be terse");
assert_eq!(messages[1]["role"], "user");
assert_eq!(messages[1]["content"], "hello");
// No Anthropic-only fields leak into the OpenAI body.
assert!(v.get("max_tokens").is_none());
assert!(v.get("system").is_none());
}
#[test]
fn openai_response_parses_fixture() {
// Shape as returned by OpenAI / vLLM /v1/chat/completions.
let body = r#"{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1719000000,
"model": "meta-llama/Llama-3.1-8B-Instruct",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": "Hi from vLLM"},
"finish_reason": "stop"
}],
"usage": {"prompt_tokens": 9, "completion_tokens": 4, "total_tokens": 13}
}"#;
assert_eq!(extract_openai_content(body).unwrap(), "Hi from vLLM");
}
#[test]
fn openai_response_errors_on_missing_choices() {
assert!(matches!(
extract_openai_content(r#"{"object":"chat.completion","choices":[]}"#),
Err(LlmError::Decode(_))
));
assert!(matches!(
extract_openai_content(r#"{"error":{"message":"invalid key"}}"#),
Err(LlmError::Decode(_))
));
}
#[test]
fn openai_uses_bearer_auth() {
let p = ChatParams {
api_key: "sk-test",
..params(Api::Openai)
};
assert_eq!(
build_headers(&p),
vec![("Authorization", "Bearer sk-test".to_string())]
);
}
#[test]
fn openai_chat_returns_content_and_no_digest() {
let canned = r#"{"choices":[{"message":{"role":"assistant","content":"ok"}}]}"#;
let client = MockClient::new(vec![Ok(canned.to_string())]);
let p = ChatParams {
api: Api::Openai,
endpoint: "http://vllm:8000/v1/chat/completions",
model: "m",
api_key: "sk",
system_prompt: "",
prompt: "hi",
};
let result = chat_with_identity(&client, &p).unwrap();
assert_eq!(result.response, "ok");
// No digest API on OpenAI-compatible endpoints — documented
// empty, and exactly one HTTP call (no probe).
assert_eq!(result.model_digest, None);
assert_eq!(client.calls.borrow().len(), 1);
}
// ---------------- Anthropic wire format ----------------
#[test]
fn anthropic_body_matches_messages_format() {
let p = ChatParams {
api: Api::Anthropic,
endpoint: "https://api.anthropic.com/v1/messages",
model: "claude-fable-5",
api_key: "sk-ant",
system_prompt: "be terse",
prompt: "hello",
};
let body = build_anthropic_body(&p);
let v: serde_json::Value = serde_json::from_str(&body).unwrap();
assert_eq!(v["model"], "claude-fable-5");
// max_tokens is mandatory in the Messages API.
assert_eq!(v["max_tokens"], 4096);
// system is a top-level field, never a message.
assert_eq!(v["system"], "be terse");
let messages = v["messages"].as_array().unwrap();
assert_eq!(messages.len(), 1);
assert_eq!(messages[0]["role"], "user");
assert_eq!(messages[0]["content"], "hello");
// No OpenAI/Ollama-only fields leak in.
assert!(v.get("stream").is_none());
}
#[test]
fn anthropic_body_omits_system_when_empty() {
let body = build_anthropic_body(&params(Api::Anthropic));
let v: serde_json::Value = serde_json::from_str(&body).unwrap();
assert!(v.get("system").is_none());
assert_eq!(v["max_tokens"], 4096);
}
#[test]
fn anthropic_response_parses_fixture() {
// Shape as returned by the Anthropic Messages API.
let body = r#"{
"id": "msg_01XFDUDYJgAACzvnptvVoYEL",
"type": "message",
"role": "assistant",
"model": "claude-fable-5",
"content": [{"type": "text", "text": "Hi from Claude"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 10, "output_tokens": 5}
}"#;
assert_eq!(extract_anthropic_content(body).unwrap(), "Hi from Claude");
}
#[test]
fn anthropic_response_errors_on_missing_content() {
assert!(matches!(
extract_anthropic_content(r#"{"type":"message","content":[]}"#),
Err(LlmError::Decode(_))
));
assert!(matches!(
extract_anthropic_content(r#"{"type":"error","error":{"message":"x"}}"#),
Err(LlmError::Decode(_))
));
}
#[test]
fn anthropic_uses_x_api_key_and_version_headers() {
let p = ChatParams {
api_key: "sk-ant-test",
..params(Api::Anthropic)
};
let headers = build_headers(&p);
assert_eq!(
headers,
vec![
("x-api-key", "sk-ant-test".to_string()),
("anthropic-version", "2023-06-01".to_string()),
]
);
// Never Bearer auth on the Anthropic path.
assert!(headers.iter().all(|(k, _)| *k != "Authorization"));
}
#[test]
fn anthropic_chat_returns_content_and_no_digest() {
let canned = r#"{"content":[{"type":"text","text":"ok"}]}"#;
let client = MockClient::new(vec![Ok(canned.to_string())]);
let p = ChatParams {
api: Api::Anthropic,
endpoint: "https://api.anthropic.com/v1/messages",
model: "claude-fable-5",
api_key: "sk-ant",
system_prompt: "",
prompt: "hi",
};
let result = chat_with_identity(&client, &p).unwrap();
assert_eq!(result.response, "ok");
assert_eq!(result.model_digest, None);
assert_eq!(client.calls.borrow().len(), 1);
}
// ---------------- digest probe (Ollama only) ----------------
#[test]
fn derive_show_url_swaps_chat_suffix() {
assert_eq!(
@ -263,10 +670,7 @@ mod tests {
let client = MockClient::new(vec![Ok(canned_show), Ok(canned_chat)]);
let p = ChatParams {
endpoint: "http://localhost:11434/api/chat",
model: "qwen",
api_key: "",
system_prompt: "",
prompt: "hi",
..params(Api::Ollama)
};
let result = chat_with_identity(&client, &p).unwrap();
assert_eq!(result.response, "the response");
@ -279,10 +683,7 @@ mod tests {
let client = MockClient::new(vec![Err(LlmError::Status(500)), Ok(canned_chat)]);
let p = ChatParams {
endpoint: "http://localhost:11434/api/chat",
model: "qwen",
api_key: "",
system_prompt: "",
prompt: "hi",
..params(Api::Ollama)
};
let result = chat_with_identity(&client, &p).unwrap();
assert_eq!(result.response, "ok");
@ -290,30 +691,26 @@ mod tests {
}
#[test]
fn chat_with_identity_skips_probe_for_non_ollama() {
fn chat_with_identity_skips_probe_for_non_ollama_url() {
let canned_chat = r#"{"message":{"content":"ok"}}"#.to_string();
let client = MockClient::new(vec![Ok(canned_chat)]);
let p = ChatParams {
endpoint: "https://api.openai.com/v1/chat/completions",
model: "gpt",
api_key: "k",
system_prompt: "",
prompt: "hi",
endpoint: "https://example.com/proxy/chat",
..params(Api::Ollama)
};
let result = chat_with_identity(&client, &p).unwrap();
assert_eq!(result.response, "ok");
assert_eq!(result.model_digest, None);
}
// ---------------- input guards ----------------
#[test]
fn empty_prompt_is_rejected() {
let client = MockClient::new(vec![]);
let p = ChatParams {
endpoint: "http://x/api/chat",
model: "x",
api_key: "",
system_prompt: "",
prompt: "",
..params(Api::Ollama)
};
assert!(matches!(
chat_with_identity(&client, &p),