flemming-it
20ff097e3b
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
The fai-module-sdk repo was renamed from `fai/module-sdk` to `fai/module-sdk-rust` to align with the three SDK families convention (base + language suffix). Pin updated accordingly. The old git.flemming.ws URL is also retired in this pass. Signed-off-by: flemming-it <sf@flemming.it> |
||
|---|---|---|
| .forgejo/workflows | ||
| src | ||
| tests | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| LICENSE | ||
| MODULE.de.md | ||
| MODULE.md | ||
| module.yaml | ||
| NOTICE | ||
| README.md | ||
| rust-toolchain.toml | ||
text-extract
F∆I module — extract plain text from PDF and DOCX documents.
Capability
| Field | Value |
|---|---|
| Capability | text.extract@0.1.0 |
| Inputs | document: bytes (PDF or DOCX) |
| Outputs | extracted: json (ExtractedDocument) |
| Permissions | none — pure in-WASM, no filesystem, no network |
| Status (in store index) | alpha |
Output schema
{
"engine": "pdf-extract",
"engine_version": "0.7",
"pages": [
{ "number": 1, "text": "..." }
]
}
confidence and bbox per page are reserved for a future
service-mode variant (host-services-backed pdfium/mupdf engine).
v0.1.0 does not populate them; the schema accepts them as
optional so that v0.2.0 can extend without an interface break.
Build
cargo build --release --target wasm32-wasip2
# Artifact: target/wasm32-wasip2/release/text_extract.wasm
The build depends on a sibling checkout of fai/module-sdk:
~/fai/
├── module_sdk/ # cloned from fai/module-sdk
└── fai_modules/
└── text-extract/ # this repo
This is a temporary path-based dependency. Once fai-module-sdk
is published to a registry or a stable git tag, this module's
Cargo.toml will switch over. The Forgejo CI workflow checks out
the SDK repo into a sibling directory before building.
Test
cargo test # 9 tests: 6 unit + 3 integration
cargo test --release # same, optimized
The DOCX path is covered end-to-end by an integration test that
builds a minimal valid DOCX in memory and feeds it through
invoke. The PDF path is covered by unit tests on the page
splitter; full PDF coverage will land once the platform's hub
loads this wasm component as part of its store-install
integration test.
Limitations of v0.1.0
- Layout-naive PDF parsing. Multi-column layouts and tables may produce reordered text. Single-column documents work reliably.
- DOCX text only. Tables, footnotes, headers/footers and embedded objects are not yet extracted. Plain paragraph text with run-level concatenation is the v0.1.0 surface.
- No OCR. Image-only PDFs return empty text per page.
An OCR variant will arrive as a separate module
(
text.ocr@x.y.z) once the host-services infrastructure for tesseract is in place.
License
Apache-2.0. See LICENSE.
Author: Dr. Stefan Flemming, Flemming.AI platform@flemming.ai Repository: https://git.flemming.ws/fai-modules/text-extract