F module - generic Ollama-compatible LLM chat adapter (llm.chat).
Find a file
Stefan Flemming a6fe962916
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
feat: sampling options (temperature, top_p, top_k, seed)
aila asked for these twice (07.08. and 08.08.) and worked around it
by baking the values into a derived Ollama model, which is the only
way they could get a reproducible pipeline.

Verified against a live Ollama: two runs with `temperature: "0"` and
`seed: "42"` return the same answer.

Each wire format gets what it actually supports, and an option the
wire does not have is REFUSED rather than dropped: `top_k` on the
OpenAI wire, `seed` on Anthropic. A silently ignored setting
produces a run that looks configured and is not, which is worse than
an error, especially for `seed`, where the whole point is being able
to repeat the run. A non-numeric value is an error too, so a typo
cannot pass as "server default".

Unset leaves the request byte-identical to 0.2.1, so no existing
caller sees a change.

Eight guards covering all three wires, both refusals, the
unset case, and that `seed` stays a whole number.

Capability 0.2.1 to 0.2.2: four optional inputs, nothing removed.

Signed-off-by: Stefan Flemming <stefan.a.flemming@googlemail.com>
2026-10-16 20:47:40 +02:00
.forgejo/workflows fix(ci): embed official.pub — round-trip verify fetched it from a private repo 2026-08-20 22:07:15 +02:00
src feat: sampling options (temperature, top_p, top_k, seed) 2026-10-16 20:47:40 +02:00
.gitignore feat: initial llm-chat v0.1.0 (llm.chat@0.1.0) 2026-05-03 23:17:36 +02:00
Cargo.lock feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
Cargo.toml feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
LICENSE chore: org rename fai-modules/fai-plugins -> chain-modules/chain-plugins + LICENSE holder 2026-06-16 18:25:36 +02:00
MODULE.de.md feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
MODULE.md feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
module.wasm feat: sampling options (temperature, top_p, top_k, seed) 2026-10-16 20:47:40 +02:00
module.yaml feat: sampling options (temperature, top_p, top_k, seed) 2026-10-16 20:47:40 +02:00
NOTICE feat: initial llm-chat v0.1.0 (llm.chat@0.1.0) 2026-05-03 23:17:36 +02:00
README.md feat: OpenAI and Anthropic wire-format adapters via new api input 2026-07-11 23:19:32 +02:00
rust-toolchain.toml feat: initial llm-chat v0.1.0 (llm.chat@0.1.0) 2026-05-03 23:17:36 +02:00

llm-chat

F∆I module providing the llm.chat capability — a generic LLM chat adapter speaking Ollama (default), OpenAI Chat Completions (OpenAI, vLLM, compatible servers), and the Anthropic Messages API, selected via the optional api input.

Capability

Field Value
Capability llm.chat@0.2.0
Inputs prompt: text, endpoint: text, model: text, api_key: text (opt), system_prompt: text (opt), api: text (opt: ollama|openai|anthropic)
Outputs response: text, model_endpoint: text, model_name: text, model_digest: text
Permissions net: localhost, net: 127.0.0.1, net: api.openai.com, net: api.anthropic.com
Status (in store index) alpha

Why a separate module from orchestrator-llm

orchestrator-llm wraps an LLM call inside a planning prompt that emits a structured F∆I Plan. llm.chat is the lower- level adapter: it sends a prompt and returns the assistant text verbatim. Flows that need a generic chat step (summarisation, translation, free-form Q&A) compose llm.chat directly; flows that need plan generation use orchestrator-llm.

Audit fields

model_endpoint, model_name, and model_digest are emitted on every successful invocation. Together they answer the audit question "which exact model produced this response?" The digest probe targets Ollama's /api/show and is best-effort — non- Ollama endpoints and transient probe failures yield an empty digest rather than blocking the response. Cf. Ch∆In docs/advanced/compliance-gaps.md Gap 1.

Build

cargo build --release --target wasm32-wasip2

Test

cargo test                       # wire-format unit tests, all host-side
cargo build --release --target wasm32-wasip2

SDK source

The Cargo.toml git-deps chain-module-sdk from https://git.flemming.ws/fai/module-sdk.git. The Forgejo instance has REQUIRE_SIGNIN_VIEW=true, so anonymous clones fail; CI uses the MODULE_SDK_PAT actions secret + git config url.X.insteadOf Y to authenticate cargo's git fetch transparently. Local dev uses the same pattern via env vars.

License

Apache-2.0.

Author: Dr. Stefan Flemming, Flemming.AI chain@flemming.ai Repository: https://git.flemming.ws/chain-modules/llm-chat