F module - generic Ollama-compatible LLM chat adapter (llm.chat).
Find a file
flemming-it 9483f196a1
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
feat: keep_alive input so a local model stays resident
Ollama unloads a model five minutes after its last request, and the
next request then pays a full reload: 30 to 40 seconds for a large
local model, long enough that an operator assumes something is broken.
The stromnetz demonstrator runs on a notebook from around 12.10. and
hits exactly that after any idle period; it works around it today by
setting OLLAMA_KEEP_ALIVE on the server, which is not expressible in a
flow.

Measured against a live Ollama: without the input the model expired
5:02 after the request, with `keep_alive: "90s"` it expired after 92
seconds. Worth knowing for callers: Ollama keeps the value for the
loaded instance, so a later request that omits keep_alive does not
fall back to five minutes until the model has actually been unloaded.

`-1` is sent as a number rather than a string, because Ollama reads
only the numeric form as "never unload"; quoted it is parsed as a
duration and the setting silently does nothing.

With any other wire format the input is REFUSED, not ignored. It
expresses an expectation about the server, and dropping it quietly
would leave the caller paying the reload while believing it was
handled. The message says what to do instead.

Capability goes 0.2.0 to 0.2.1: the addition is optional and additive,
so `@^0.2` and `@^0` both keep resolving, and an unset value leaves
the request byte-identical to the previous version.

Five guards, including that unset omits the field entirely and that
the other wire formats never carry it. Mutation-checked three ways.

Signed-off-by: flemming-it <sf@flemming.it>
2026-10-07 05:49:16 +02:00
.forgejo/workflows fix(ci): embed official.pub — round-trip verify fetched it from a private repo 2026-08-20 22:07:15 +02:00
src feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
.gitignore feat: initial llm-chat v0.1.0 (llm.chat@0.1.0) 2026-05-03 23:17:36 +02:00
Cargo.lock feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
Cargo.toml feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
LICENSE chore: org rename fai-modules/fai-plugins -> chain-modules/chain-plugins + LICENSE holder 2026-06-16 18:25:36 +02:00
MODULE.de.md feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
MODULE.md feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
module.wasm feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
module.yaml feat: keep_alive input so a local model stays resident 2026-10-07 05:49:16 +02:00
NOTICE feat: initial llm-chat v0.1.0 (llm.chat@0.1.0) 2026-05-03 23:17:36 +02:00
README.md feat: OpenAI and Anthropic wire-format adapters via new api input 2026-07-11 23:19:32 +02:00
rust-toolchain.toml feat: initial llm-chat v0.1.0 (llm.chat@0.1.0) 2026-05-03 23:17:36 +02:00

llm-chat

F∆I module providing the llm.chat capability — a generic LLM chat adapter speaking Ollama (default), OpenAI Chat Completions (OpenAI, vLLM, compatible servers), and the Anthropic Messages API, selected via the optional api input.

Capability

Field Value
Capability llm.chat@0.2.0
Inputs prompt: text, endpoint: text, model: text, api_key: text (opt), system_prompt: text (opt), api: text (opt: ollama|openai|anthropic)
Outputs response: text, model_endpoint: text, model_name: text, model_digest: text
Permissions net: localhost, net: 127.0.0.1, net: api.openai.com, net: api.anthropic.com
Status (in store index) alpha

Why a separate module from orchestrator-llm

orchestrator-llm wraps an LLM call inside a planning prompt that emits a structured F∆I Plan. llm.chat is the lower- level adapter: it sends a prompt and returns the assistant text verbatim. Flows that need a generic chat step (summarisation, translation, free-form Q&A) compose llm.chat directly; flows that need plan generation use orchestrator-llm.

Audit fields

model_endpoint, model_name, and model_digest are emitted on every successful invocation. Together they answer the audit question "which exact model produced this response?" The digest probe targets Ollama's /api/show and is best-effort — non- Ollama endpoints and transient probe failures yield an empty digest rather than blocking the response. Cf. Ch∆In docs/advanced/compliance-gaps.md Gap 1.

Build

cargo build --release --target wasm32-wasip2

Test

cargo test                       # wire-format unit tests, all host-side
cargo build --release --target wasm32-wasip2

SDK source

The Cargo.toml git-deps chain-module-sdk from https://git.flemming.ws/fai/module-sdk.git. The Forgejo instance has REQUIRE_SIGNIN_VIEW=true, so anonymous clones fail; CI uses the MODULE_SDK_PAT actions secret + git config url.X.insteadOf Y to authenticate cargo's git fetch transparently. Local dev uses the same pattern via env vars.

License

Apache-2.0.

Author: Dr. Stefan Flemming, Flemming.AI chain@flemming.ai Repository: https://git.flemming.ws/chain-modules/llm-chat