|
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
Ollama unloads a model five minutes after its last request, and the next request then pays a full reload: 30 to 40 seconds for a large local model, long enough that an operator assumes something is broken. The stromnetz demonstrator runs on a notebook from around 12.10. and hits exactly that after any idle period; it works around it today by setting OLLAMA_KEEP_ALIVE on the server, which is not expressible in a flow. Measured against a live Ollama: without the input the model expired 5:02 after the request, with `keep_alive: "90s"` it expired after 92 seconds. Worth knowing for callers: Ollama keeps the value for the loaded instance, so a later request that omits keep_alive does not fall back to five minutes until the model has actually been unloaded. `-1` is sent as a number rather than a string, because Ollama reads only the numeric form as "never unload"; quoted it is parsed as a duration and the setting silently does nothing. With any other wire format the input is REFUSED, not ignored. It expresses an expectation about the server, and dropping it quietly would leave the caller paying the reload while believing it was handled. The message says what to do instead. Capability goes 0.2.0 to 0.2.1: the addition is optional and additive, so `@^0.2` and `@^0` both keep resolving, and an unset value leaves the request byte-identical to the previous version. Five guards, including that unset omits the field entirely and that the other wire formats never carry it. Mutation-checked three ways. Signed-off-by: flemming-it <sf@flemming.it> |
||
|---|---|---|
| .forgejo/workflows | ||
| src | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| LICENSE | ||
| MODULE.de.md | ||
| MODULE.md | ||
| module.wasm | ||
| module.yaml | ||
| NOTICE | ||
| README.md | ||
| rust-toolchain.toml | ||
llm-chat
F∆I module providing the llm.chat capability — a generic
LLM chat adapter speaking Ollama (default), OpenAI Chat
Completions (OpenAI, vLLM, compatible servers), and the
Anthropic Messages API, selected via the optional api input.
Capability
| Field | Value |
|---|---|
| Capability | llm.chat@0.2.0 |
| Inputs | prompt: text, endpoint: text, model: text, api_key: text (opt), system_prompt: text (opt), api: text (opt: ollama|openai|anthropic) |
| Outputs | response: text, model_endpoint: text, model_name: text, model_digest: text |
| Permissions | net: localhost, net: 127.0.0.1, net: api.openai.com, net: api.anthropic.com |
| Status (in store index) | alpha |
Why a separate module from orchestrator-llm
orchestrator-llm wraps an LLM call inside a planning prompt
that emits a structured F∆I Plan. llm.chat is the lower-
level adapter: it sends a prompt and returns the assistant text
verbatim. Flows that need a generic chat step (summarisation,
translation, free-form Q&A) compose llm.chat directly; flows
that need plan generation use orchestrator-llm.
Audit fields
model_endpoint, model_name, and model_digest are emitted
on every successful invocation. Together they answer the audit
question "which exact model produced this response?" The digest
probe targets Ollama's /api/show and is best-effort — non-
Ollama endpoints and transient probe failures yield an empty
digest rather than blocking the response. Cf. Ch∆In
docs/advanced/compliance-gaps.md Gap 1.
Build
cargo build --release --target wasm32-wasip2
Test
cargo test # wire-format unit tests, all host-side
cargo build --release --target wasm32-wasip2
SDK source
The Cargo.toml git-deps chain-module-sdk from
https://git.flemming.ws/fai/module-sdk.git. The Forgejo
instance has REQUIRE_SIGNIN_VIEW=true, so anonymous clones
fail; CI uses the MODULE_SDK_PAT actions secret + git config url.X.insteadOf Y to authenticate cargo's git fetch
transparently. Local dev uses the same pattern via env vars.
License
Apache-2.0.
Author: Dr. Stefan Flemming, Flemming.AI chain@flemming.ai Repository: https://git.flemming.ws/chain-modules/llm-chat