flemming-it
4c10a5ff3d
All checks were successful
CI / Linux x86_64 (Forgejo) (push) Successful in 1m57s
Generic Ollama-compatible LLM chat adapter. The lower-level
counterpart to orchestrator-llm: orchestrator-llm wraps the LLM
in a planning prompt that emits a structured F∆I Plan; this
module is the plain-prompt adapter that flows compose for
summarisation, translation, free-form Q&A, etc.
Capability surface:
Inputs:
prompt : text
endpoint : text (Ollama /api/chat URL)
model : text
api_key : text (optional bearer token)
system_prompt : text (optional)
Outputs:
response : text (assistant reply)
model_endpoint : text (audit correlation)
model_name : text (audit correlation)
model_digest : text (Ollama /api/show probe; empty
for non-Ollama or transient failures)
Permissions: net to localhost / 127.0.0.1 / api.openai.com /
api.anthropic.com.
The audit-field trio (endpoint + name + digest) closes the same
forensic gap that orchestrator-llm v0.3.1 closed for plan
generation: any historical chat invocation can be traced to the
exact model that produced it.
Implementation reuses the same defensive Ollama client pattern
from orchestrator-llm — derive_show_url + extract_show_digest
+ best-effort probe_model_digest. Duplication accepted at the
two-module mark; a shared crate refactor lands once a third
module needs the same plumbing.
12 host-side tests cover prompt building, Ollama-shaped
response parsing, URL transform, digest extraction (top-level
+ nested), end-to-end success, end-to-end probe-failure
swallow, end-to-end skip-for-non-Ollama, and the missing-input
guards.
Wasm artifact: 294 KB. Verified to build with v1.0 fai:platform
imports baked in.
Bootstrapped via 'fai new module llm.chat' (workspace v0.10.13)
which now produces an SDK-based template directly.
Signed-off-by: flemming-it <sf@flemming.it>
113 lines
3.3 KiB
Rust
113 lines
3.3 KiB
Rust
//! `llm.chat` — generic Ollama-compatible LLM chat adapter.
|
|
//!
|
|
//! Sends a single user prompt (with optional system prompt) to an
|
|
//! Ollama `/api/chat` endpoint and returns the assistant text.
|
|
//! Reports `model_endpoint`, `model_name`, and `model_digest` as
|
|
//! separate outputs for audit consumers.
|
|
//!
|
|
//! v0.1.0 targets Ollama only. OpenAI / Anthropic adapters land
|
|
//! when a flow needs them.
|
|
|
|
mod llm;
|
|
|
|
use fai_module_sdk::prelude::*;
|
|
|
|
#[fai_module]
|
|
pub fn invoke(_ctx: Context, inputs: Inputs) -> Result<Outputs, ModuleError> {
|
|
let prompt = inputs.require_text("prompt")?.to_string();
|
|
let endpoint = inputs.require_text("endpoint")?.to_string();
|
|
let model = inputs.require_text("model")?.to_string();
|
|
let api_key = inputs
|
|
.get("api_key")
|
|
.and_then(payload_text)
|
|
.unwrap_or_default();
|
|
let system_prompt = inputs
|
|
.get("system_prompt")
|
|
.and_then(payload_text)
|
|
.unwrap_or_default();
|
|
|
|
let client = make_client();
|
|
let params = crate::llm::ChatParams {
|
|
endpoint: &endpoint,
|
|
model: &model,
|
|
api_key: &api_key,
|
|
system_prompt: &system_prompt,
|
|
prompt: &prompt,
|
|
};
|
|
let result = crate::llm::chat_with_identity(&client, ¶ms)
|
|
.map_err(|e| ModuleError::internal(e.to_string()))?;
|
|
|
|
Ok(Outputs::new()
|
|
.with_text("response", result.response)
|
|
.with_text("model_endpoint", endpoint)
|
|
.with_text("model_name", model)
|
|
.with_text("model_digest", result.model_digest.unwrap_or_default()))
|
|
}
|
|
|
|
fn payload_text(p: &Payload) -> Option<String> {
|
|
match p {
|
|
Payload::Text(s) => Some(s.clone()),
|
|
_ => None,
|
|
}
|
|
}
|
|
|
|
#[cfg(target_arch = "wasm32")]
|
|
fn make_client() -> WakiClient {
|
|
WakiClient
|
|
}
|
|
|
|
#[cfg(not(target_arch = "wasm32"))]
|
|
fn make_client() -> HostStubClient {
|
|
HostStubClient
|
|
}
|
|
|
|
#[cfg(not(target_arch = "wasm32"))]
|
|
struct HostStubClient;
|
|
|
|
#[cfg(not(target_arch = "wasm32"))]
|
|
#[allow(dead_code)]
|
|
impl crate::llm::LlmClient for HostStubClient {
|
|
fn post_json(
|
|
&self,
|
|
_url: &str,
|
|
_body: &str,
|
|
_api_key: &str,
|
|
) -> Result<String, crate::llm::LlmError> {
|
|
Err(crate::llm::LlmError::Http(
|
|
"LLM HTTP path is unavailable on the host build; only wasm32 supports outbound HTTP"
|
|
.to_string(),
|
|
))
|
|
}
|
|
}
|
|
|
|
#[cfg(target_arch = "wasm32")]
|
|
struct WakiClient;
|
|
|
|
#[cfg(target_arch = "wasm32")]
|
|
impl crate::llm::LlmClient for WakiClient {
|
|
fn post_json(
|
|
&self,
|
|
url: &str,
|
|
body: &str,
|
|
api_key: &str,
|
|
) -> Result<String, crate::llm::LlmError> {
|
|
let mut request = waki::Client::new()
|
|
.post(url)
|
|
.header("Content-Type", "application/json")
|
|
.body(body.to_string());
|
|
if !api_key.is_empty() {
|
|
request = request.header("Authorization", &format!("Bearer {api_key}"));
|
|
}
|
|
let response = request
|
|
.send()
|
|
.map_err(|e| crate::llm::LlmError::Http(e.to_string()))?;
|
|
let status = response.status_code();
|
|
if !(200..300).contains(&status) {
|
|
return Err(crate::llm::LlmError::Status(status));
|
|
}
|
|
let bytes = response
|
|
.body()
|
|
.map_err(|e| crate::llm::LlmError::Http(e.to_string()))?;
|
|
String::from_utf8(bytes).map_err(|e| crate::llm::LlmError::Decode(e.to_string()))
|
|
}
|
|
}
|