flemming-it
cc72db0421
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 2s
First real F∆I capability module. Ships as alpha in the store
index — installable, but layout-naive PDF extraction. Bumps to
1.0.0 require either the planned service-mode variant
(host-services-backed pdfium/mupdf) or confirmed-good
extraction quality on a representative document set.
Capability surface:
Inputs: document : bytes (PDF or DOCX)
Outputs: extracted : json (ExtractedDocument)
Permissions: none — pure in-WASM, no filesystem, no network
Output schema:
{
"engine": "pdf-extract" | "zip+quick-xml",
"engine_version": "<version>",
"pages": [{ "number": <u32>, "text": <string>,
"confidence?": <f64>, "bbox?": {x,y,w,h} }]
}
The optional confidence and bbox fields are reserved for the
future service-mode variant. The host-services infrastructure
that backs pdfium/mupdf does not exist yet (Phase 0.5+); when
it lands, text-extract v0.2.0 will populate these fields without
an API break.
Engines:
PDF: pdf-extract 0.7 (pure Rust, MIT-licensed). Single-column
documents extract reliably. Multi-column or table-heavy
layouts may produce reordered text — documented in
README as a known limitation of the in-WASM v0.1.0 path.
DOCX: ad-hoc parser using the zip and quick-xml crates. Walks
word/document.xml, concatenates run-level text, inserts a
newline at each paragraph boundary. Tables, footnotes,
and embedded objects are not extracted in v0.1.0.
The module is the first reference user of fai-module-sdk. The
public surface is one #[fai_module] function — no wit_bindgen,
no Guest impl, no allow attributes. Testing surface is the same
function via crate-type = ["cdylib", "rlib"]; the SDK macro
gates its WASM glue on target_arch = "wasm32" so host integration
tests work out of the box.
Tests: 6 unit + 3 integration, all green. The DOCX integration
test builds a minimal valid DOCX in memory and verifies the
JSON output schema. PDF integration coverage will land in a
follow-up once a representative test fixture is bundled.
The Forgejo CI workflow clones fai/module-sdk into a sibling
location to satisfy the path-based dev dependency. The path
will switch to a versioned git tag once SDK releases stabilize.
Signed-off-by: flemming-it <sf@flemming.it>
51 lines
1.3 KiB
TOML
51 lines
1.3 KiB
TOML
# Standalone Cargo.toml — targets wasm32-wasip2.
|
|
#
|
|
# Build with:
|
|
# cargo build --release --target wasm32-wasip2
|
|
|
|
[package]
|
|
name = "text_extract"
|
|
version = "0.1.0"
|
|
edition = "2024"
|
|
authors = ["Dr. Stefan Flemming <platform@flemming.ai>"]
|
|
license = "Apache-2.0"
|
|
publish = false
|
|
description = "F∆I module — extract plain text from PDF and DOCX documents"
|
|
repository = "https://git.flemming.ws/fai-modules/text-extract"
|
|
rust-version = "1.85"
|
|
|
|
[lib]
|
|
crate-type = ["cdylib", "rlib"]
|
|
|
|
[dependencies]
|
|
fai-module-sdk = { path = "../../module_sdk/crates/fai-module-sdk" }
|
|
serde = { version = "1", features = ["derive"] }
|
|
pdf-extract = "0.7"
|
|
zip = { version = "2", default-features = false, features = ["deflate"] }
|
|
quick-xml = "0.36"
|
|
|
|
[dev-dependencies]
|
|
serde_json = "1"
|
|
# Restricted features: deflate64 (a default feature of zip 2.x)
|
|
# requires the unstable `unbounded_shifts` library feature.
|
|
# We only need DEFLATE for round-tripping a minimal DOCX in tests.
|
|
zip = { version = "2", default-features = false, features = ["deflate"] }
|
|
|
|
[profile.release]
|
|
opt-level = "s"
|
|
lto = true
|
|
codegen-units = 1
|
|
strip = "symbols"
|
|
|
|
[lints.rust]
|
|
unsafe_op_in_unsafe_fn = "warn"
|
|
|
|
[lints.clippy]
|
|
unwrap_used = "deny"
|
|
expect_used = "deny"
|
|
panic = "deny"
|
|
print_stdout = "deny"
|
|
print_stderr = "deny"
|
|
|
|
[workspace]
|
|
# Empty workspace table so this package is NOT picked up by a parent.
|