flemming-it
62e0d0b7b7
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 1s
Signed-off-by: flemming-it <sf@flemming.it>
85 lines
2.5 KiB
Markdown
85 lines
2.5 KiB
Markdown
# text-extract
|
|
|
|
F∆I module — extract plain text from PDF and DOCX documents.
|
|
|
|
## Capability
|
|
|
|
| Field | Value |
|
|
|-------|-------|
|
|
| Capability | `text.extract@0.1.0` |
|
|
| Inputs | `document: bytes` (PDF or DOCX) |
|
|
| Outputs | `extracted: json` (ExtractedDocument) |
|
|
| Permissions | _none_ — pure in-WASM, no filesystem, no network |
|
|
| Status (in store index) | `alpha` |
|
|
|
|
## Output schema
|
|
|
|
```json
|
|
{
|
|
"engine": "pdf-extract",
|
|
"engine_version": "0.7",
|
|
"pages": [
|
|
{ "number": 1, "text": "..." }
|
|
]
|
|
}
|
|
```
|
|
|
|
`confidence` and `bbox` per page are reserved for a future
|
|
service-mode variant (host-services-backed pdfium/mupdf engine).
|
|
v0.1.0 does not populate them; the schema accepts them as
|
|
optional so that v0.2.0 can extend without an interface break.
|
|
|
|
## Build
|
|
|
|
```bash
|
|
cargo build --release --target wasm32-wasip2
|
|
# Artifact: target/wasm32-wasip2/release/text_extract.wasm
|
|
```
|
|
|
|
The build depends on a sibling checkout of `fai/module-sdk`:
|
|
|
|
```
|
|
~/fai/
|
|
├── module_sdk/ # cloned from fai/module-sdk
|
|
└── fai_modules/
|
|
└── text-extract/ # this repo
|
|
```
|
|
|
|
This is a temporary path-based dependency. Once `chain-module-sdk`
|
|
is published to a registry or a stable git tag, this module's
|
|
`Cargo.toml` will switch over. The Forgejo CI workflow checks out
|
|
the SDK repo into a sibling directory before building.
|
|
|
|
## Test
|
|
|
|
```bash
|
|
cargo test # 9 tests: 6 unit + 3 integration
|
|
cargo test --release # same, optimized
|
|
```
|
|
|
|
The DOCX path is covered end-to-end by an integration test that
|
|
builds a minimal valid DOCX in memory and feeds it through
|
|
`invoke`. The PDF path is covered by unit tests on the page
|
|
splitter; full PDF coverage will land once the platform's hub
|
|
loads this wasm component as part of its store-install
|
|
integration test.
|
|
|
|
## Limitations of v0.1.0
|
|
|
|
- **Layout-naive PDF parsing.** Multi-column layouts and tables
|
|
may produce reordered text. Single-column documents work
|
|
reliably.
|
|
- **DOCX text only.** Tables, footnotes, headers/footers and
|
|
embedded objects are not yet extracted. Plain paragraph text
|
|
with run-level concatenation is the v0.1.0 surface.
|
|
- **No OCR.** Image-only PDFs return empty text per page.
|
|
An OCR variant will arrive as a separate module
|
|
(`text.ocr@x.y.z`) once the host-services infrastructure
|
|
for tesseract is in place.
|
|
|
|
## License
|
|
|
|
Apache-2.0. See `LICENSE`.
|
|
|
|
Author: Dr. Stefan Flemming, Flemming.AI <chain@flemming.ai>
|
|
Repository: https://git.flemming.ws/fai-modules/text-extract
|