flemming-it
3e018f528a
Some checks failed
CI / Linux x86_64 (Forgejo) (push) Failing after 1m20s
NER anonymization of German legal texts (plain text + DOCX) via the JuraNER model by Harshil Darji (MIT). The transformer model runs in the judge-ner host service; this module forwards documents over loopback HTTP and maps responses onto the capability contract.
112 lines
4 KiB
YAML
112 lines
4 KiB
YAML
schema_version: 3
|
|
provider: judge
|
|
|
|
name: text-anonymize-legal-de
|
|
version: 0.1.0
|
|
|
|
# Bridge module: forwards the document to the judge-ner host service
|
|
# (FastAPI, JuraNER transformers model) over loopback HTTP and maps
|
|
# the response onto the frozen capability contract below.
|
|
provides:
|
|
- capability: text.anonymize.legal-de
|
|
version: 0.1.0
|
|
|
|
inputs:
|
|
request:
|
|
type: json
|
|
description:
|
|
en: |
|
|
Anonymization request:
|
|
{
|
|
"mode": "text" | "docx",
|
|
"options": { // all optional
|
|
"importance_levels": ["High"], // High|Mid|Low
|
|
"confidence_threshold": 0.8, // 0.0..1.0
|
|
"manual_phrases": [], // extra phrases to redact
|
|
"maintain_consistency": true, // same surface -> same token
|
|
"remove_rubrum": false // drop the rubrum block
|
|
}
|
|
}
|
|
de: |
|
|
Anonymisierungs-Anfrage:
|
|
{
|
|
"mode": "text" | "docx",
|
|
"options": { // alle optional
|
|
"importance_levels": ["High"], // High|Mid|Low
|
|
"confidence_threshold": 0.8, // 0.0..1.0
|
|
"manual_phrases": [], // zusätzlich zu schwärzende Phrasen
|
|
"maintain_consistency": true, // gleiche Oberfläche -> gleiches Token
|
|
"remove_rubrum": false // Rubrum-Block entfernen
|
|
}
|
|
}
|
|
document:
|
|
type: bytes
|
|
description:
|
|
en: |
|
|
The document to anonymize. UTF-8 plain text for mode=text,
|
|
a DOCX file for mode=docx.
|
|
de: |
|
|
Das zu anonymisierende Dokument. UTF-8-Klartext bei mode=text,
|
|
eine DOCX-Datei bei mode=docx.
|
|
endpoint:
|
|
type: text
|
|
description:
|
|
en: |
|
|
Optional base URL of the judge-ner host service.
|
|
Default: "http://127.0.0.1:8756".
|
|
de: |
|
|
Optionale Basis-URL des judge-ner-Host-Services.
|
|
Default: "http://127.0.0.1:8756".
|
|
|
|
outputs:
|
|
result:
|
|
type: json
|
|
description:
|
|
en: |
|
|
{
|
|
"replacements": [ { "text", "label", "label_description",
|
|
"start", "end", "score", "line_number",
|
|
"anonymized", ...run-id fields for docx }, ... ],
|
|
"text": "..." // anonymized text, only for mode=text
|
|
}
|
|
`replacements` lists all detected entities; entries with
|
|
"anonymized": null were detected but not replaced. A
|
|
"statistics" object (and "rubrum" when remove_rubrum matched)
|
|
is passed through from the service.
|
|
de: |
|
|
{
|
|
"replacements": [ { "text", "label", "label_description",
|
|
"start", "end", "score", "line_number",
|
|
"anonymized", ...Run-ID-Felder bei docx }, ... ],
|
|
"text": "..." // anonymisierter Text, nur bei mode=text
|
|
}
|
|
`replacements` enthält alle erkannten Entitäten; Einträge mit
|
|
"anonymized": null wurden erkannt, aber nicht ersetzt. Ein
|
|
"statistics"-Objekt (und "rubrum" bei erkanntem Rubrum) wird
|
|
vom Service durchgereicht.
|
|
document:
|
|
type: bytes
|
|
description:
|
|
en: |
|
|
The anonymized document: the anonymized UTF-8 text for
|
|
mode=text, the rewritten DOCX for mode=docx.
|
|
de: |
|
|
Das anonymisierte Dokument: bei mode=text die anonymisierte
|
|
UTF-8-Textfassung, bei mode=docx das umgeschriebene DOCX.
|
|
|
|
# MIME allow-list for the bytes input (Studio file-picker filter).
|
|
accepts_mime:
|
|
- text/plain
|
|
- application/vnd.openxmlformats-officedocument.wordprocessingml.document
|
|
|
|
# Outbound HTTP to the local judge-ner host service only.
|
|
permissions:
|
|
- "net: localhost"
|
|
- "net: 127.0.0.1"
|
|
|
|
# The hub refuses to dispatch a step to this module unless the
|
|
# operator declared a `judge-ner` entry in the service registry
|
|
# (~/.chain/config.yaml `services:`). See
|
|
# fai_chain/docs/architecture/host-services.md.
|
|
requires_services:
|
|
- judge-ner
|