feat: text.anonymize v0.1.0 — regex-based PII redaction
First release. Pure-Rust, in-WASM, regex-only, declares no
permissions. Suitable as a first redaction pass after
text.extract before any cloud-LLM step.
Detection categories:
EMAIL RFC-5321-ish local@domain, IDN-aware.
PHONE International (+CC …) and DE national
(030 …, 0151-…) shapes, 7..20 raw digits.
IBAN Word-bounded [A-Z]{2}\d{2}[A-Z0-9]{11,30}.
Structural only — MOD-97 checksum
deliberately skipped so partial / truncated
tokens in running text still get redacted.
BIC 8 or 11 uppercase alnum.
IPV4 Four 0..255 octets, dot-separated.
GERMAN_TAX_ID 11 consecutive digits, word-bounded.
CUSTOM Operator-supplied bare terms from the
newline-separated `custom_terms` input,
matched whole-word case-insensitive.
Token shape: ⟦TYPE_N⟧ — U+27E6 / U+27E7 mathematical white
square brackets. Distinct from any plain ASCII `[…]` already
present in source text (Markdown links, legal citations,
code blocks) so a reviewer never has to guess which `[…]`
is a redaction.
Outputs:
anonymized text Input with PII replaced by ⟦TYPE_N⟧.
Counter restarts at 1 per type so the
tokens stay operator-readable.
report json { redactions: [{type, token, original,
offset}…], counts: { TYPE: n, … } }.
Full original-text reconstruction is
possible from this — the GDPR
Art. 32(1)(a) "ability to undo"
requirement.
Quality bar (7 unit tests):
* email round-trip
* IBAN + BIC don't eat each other
* three phone-number shapes redact
* IPv4 only matches valid 0..255 octets
* custom_terms case-insensitive
* no double-redaction on overlapping patterns
* per-category counter resets correctly
Built artefact: target/wasm32-wasip2/release/text_anonymize.wasm
(~180 KiB stripped).
NER for free-text names / organisations / locations is the
v0.2.0 plan once a benchmarked ONNX model is selected; the
operator's `custom_terms` field is the v0.1.0 escape hatch.
Signed-off-by: flemming-it <sf@flemming.it>
This commit is contained in:
commit
734d8f6e6f
8 changed files with 1164 additions and 0 deletions
53
module.yaml
Normal file
53
module.yaml
Normal file
|
|
@ -0,0 +1,53 @@
|
|||
schema_version: 1
|
||||
name: text-anonymize
|
||||
version: 0.1.0
|
||||
|
||||
# Capability provided by this module.
|
||||
provides:
|
||||
- capability: text.anonymize
|
||||
version: 0.1.0
|
||||
|
||||
# Inputs the invoke function accepts.
|
||||
inputs:
|
||||
# The text to anonymize. Plain UTF-8, any length the host
|
||||
# tolerates as a Payload::Text (operator policy caps).
|
||||
text: text
|
||||
# Optional newline-separated list of additional bare terms
|
||||
# (operator-supplied — names of people, organisations, or
|
||||
# places that the regex patterns won't catch) to redact in
|
||||
# addition to the built-in categories. Empty string disables
|
||||
# this extension. Each line is matched whole-word
|
||||
# (case-insensitive).
|
||||
custom_terms: text
|
||||
|
||||
# Outputs produced.
|
||||
outputs:
|
||||
# The anonymized text. Each detected entity is replaced
|
||||
# with a token of the shape ⟦TYPE_N⟧ where TYPE is the
|
||||
# category (EMAIL, PHONE, IBAN, BIC, IPV4, GERMAN_TAX_ID,
|
||||
# NAME, CUSTOM) and N is a stable counter starting at 1
|
||||
# per category, in order of first occurrence. The opening
|
||||
# / closing characters are U+27E6 / U+27E7 (mathematical
|
||||
# white square brackets) so the token never collides with
|
||||
# plain `[…]` already present in the source.
|
||||
anonymized: text
|
||||
# JSON report describing each redaction:
|
||||
# {
|
||||
# "redactions": [
|
||||
# { "type": "EMAIL", "token": "⟦EMAIL_1⟧",
|
||||
# "original": "foo@bar.de", "offset": 42 }
|
||||
# ],
|
||||
# "counts": { "EMAIL": 3, "PHONE": 1, ... }
|
||||
# }
|
||||
# The full `original` text is included so the operator can
|
||||
# run a verify-pass against the redacted file (compliance:
|
||||
# GDPR Art. 32(1)(a) "ability to undo" requirement). Hub
|
||||
# operators that want a pseudo-anonymisation-only flow can
|
||||
# discard this output downstream.
|
||||
report: json
|
||||
|
||||
# Permissions required.
|
||||
#
|
||||
# Pure-Rust, regex-only, in-WASM. No filesystem, no network,
|
||||
# no LLM. The empty list makes that explicit.
|
||||
permissions: []
|
||||
Loading…
Add table
Add a link
Reference in a new issue