chore(security): externalise confidentiality term list
Moves the list of private organisation / pilot / codename
strings the security gate blocks OUT of the repo entirely.
Before: tools/security/check-staged.sh + the Today AI prompt
+ docs/today-pipeline.md held the strings in plaintext. The
whole point of the gate is to keep certain strings out of
committed artefacts, so holding them in a committed
artefact was self-defeating — anyone with read access to
the repo trivially recovered the very list we tried to
protect.
After:
- The gate reads a runtime file
`${FAI_BANNED_TERMS_FILE:-~/.fai-security/banned-terms.txt}`
at scan time. One regex per line, `#`-prefixed comments,
matched case-insensitively against staged diffs and
commit messages. Repos contain no copy.
- Pre-commit / commit-msg modes log a warning + skip the
confidential-terms scan if the file is missing (fresh
checkouts shouldn't trip until the operator bootstraps
the list).
- CI mode (`check-staged.sh ci`) FAILS when the file is
missing — runners are expected to be bootstrapped by
their deploy step.
- The unit-test harness uses a synthetic placeholder term
(`SYNTHETIC_BANNED_TERM_XYZZY`) injected via a temp
banned-terms file, so the test never references real
customer names.
- docs/today-pipeline.md + tools/today/prompt.template.md
point at the runtime file instead of enumerating terms.
Operator bootstrap (one-time, per machine):
mkdir -p ~/.fai-security && chmod 700 ~/.fai-security
printf '\\b%s\\b\\n' TERM_A TERM_B > ~/.fai-security/banned-terms.txt
chmod 600 ~/.fai-security/banned-terms.txt
Gate self-tests: 18 passed, 0 failed.
Signed-off-by: flemming-it <sf@flemming.it>
Signed-off-by: flemming-it <stefan.a.flemming@googlemail.com>
This commit is contained in:
parent
21c2a0f411
commit
efa9871a75
4 changed files with 84 additions and 20 deletions
|
|
@ -83,9 +83,12 @@ The prompt template (`tools/today/prompt.template.md`) hard-encodes:
|
||||||
`feedback_no_marketing_speak.md` is included in the prompt: ban
|
`feedback_no_marketing_speak.md` is included in the prompt: ban
|
||||||
"viral", "killer", "powerful", "just works", "revolutionary",
|
"viral", "killer", "powerful", "just works", "revolutionary",
|
||||||
"game-changing".
|
"game-changing".
|
||||||
4. **No private references.** Generic terms only; no
|
4. **No private references.** Generic terms only — no pilot
|
||||||
ITDZ / Kammergericht / HTW / J∆I per
|
customer names, internal codenames, or specific
|
||||||
`feedback_confidentiality.md`.
|
institutions. The operator's banned-terms list lives at
|
||||||
|
`~/.fai-security/banned-terms.txt` (outside the repo on
|
||||||
|
purpose; gated by the pre-commit hook in
|
||||||
|
`tools/security/check-staged.sh`).
|
||||||
5. **Concrete CTA.** Every story names exactly one action the operator
|
5. **Concrete CTA.** Every story names exactly one action the operator
|
||||||
can take in Studio (which page, which button).
|
can take in Studio (which page, which button).
|
||||||
6. **Body fits in ~3 sentences.** Hero card height is fixed; longer
|
6. **Body fits in ~3 sentences.** Hero card height is fixed; longer
|
||||||
|
|
|
||||||
|
|
@ -63,16 +63,49 @@ FILE_ALLOWLIST=(
|
||||||
'fixtures?/.*\.(pem|key)$'
|
'fixtures?/.*\.(pem|key)$'
|
||||||
)
|
)
|
||||||
|
|
||||||
# Memory: confidentiality.md — these are PRIVATE. Never in public
|
# Confidential terms are loaded at runtime from a file outside
|
||||||
# code / docs / commits / examples. Word-boundary matched to avoid
|
# the repo (~/.fai-security/banned-terms.txt by default; override
|
||||||
# false-positives on substrings.
|
# with $FAI_BANNED_TERMS_FILE). The list itself is private — the
|
||||||
CONFIDENTIAL_TERMS=(
|
# whole point of the gate is to keep certain strings out of any
|
||||||
'\bITDZ\b'
|
# committed artefact, so the list of strings to gate on must not
|
||||||
'\bKammergericht\b'
|
# live in a committed artefact either.
|
||||||
'\bHTW\b'
|
#
|
||||||
'J∆I'
|
# File format: one regex per line. Lines starting with `#` and
|
||||||
'\bJAI\b'
|
# blank lines are ignored. Each line is matched as a
|
||||||
)
|
# case-insensitive grep -E pattern with `-i`; use `\b…\b` for
|
||||||
|
# word boundaries.
|
||||||
|
#
|
||||||
|
# Bootstrap on a fresh machine:
|
||||||
|
# mkdir -p ~/.fai-security
|
||||||
|
# chmod 700 ~/.fai-security
|
||||||
|
# printf '\\b%s\\b\\n' TERM_A TERM_B > ~/.fai-security/banned-terms.txt
|
||||||
|
# chmod 600 ~/.fai-security/banned-terms.txt
|
||||||
|
#
|
||||||
|
# When the file is missing the gate logs a warning but does NOT
|
||||||
|
# fail — running on a fresh checkout where the operator hasn't
|
||||||
|
# yet bootstrapped the list shouldn't block them. Real CI runs
|
||||||
|
# explicitly verify the file is present and refuse otherwise
|
||||||
|
# (see scan_ci_mode below).
|
||||||
|
CONFIDENTIAL_TERMS=()
|
||||||
|
_load_confidential_terms() {
|
||||||
|
local file="${FAI_BANNED_TERMS_FILE:-$HOME/.fai-security/banned-terms.txt}"
|
||||||
|
if [ ! -f "$file" ]; then
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
while IFS= read -r line; do
|
||||||
|
# Strip trailing comments + leading/trailing whitespace.
|
||||||
|
line="${line%%#*}"
|
||||||
|
line="${line#"${line%%[![:space:]]*}"}"
|
||||||
|
line="${line%"${line##*[![:space:]]}"}"
|
||||||
|
[ -z "$line" ] && continue
|
||||||
|
CONFIDENTIAL_TERMS+=("$line")
|
||||||
|
done < "$file"
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
if ! _load_confidential_terms; then
|
||||||
|
printf "%s⚠ confidentiality list not found at ${FAI_BANNED_TERMS_FILE:-~/.fai-security/banned-terms.txt}; gate will skip the confidential-terms scan%s\n" \
|
||||||
|
"${YEL:-}" "${RST:-}" >&2
|
||||||
|
fi
|
||||||
|
|
||||||
# Memory: feedback_no_marketing_speak.md.
|
# Memory: feedback_no_marketing_speak.md.
|
||||||
BANNED_PHRASES=(
|
BANNED_PHRASES=(
|
||||||
|
|
@ -142,7 +175,10 @@ scan_diff() {
|
||||||
local diff="$1"
|
local diff="$1"
|
||||||
[ -z "$diff" ] && return
|
[ -z "$diff" ] && return
|
||||||
scan_secrets "$diff"
|
scan_secrets "$diff"
|
||||||
scan_terms_ci "confidential term" "$diff" "${CONFIDENTIAL_TERMS[@]}"
|
# `${ARR[@]+"${ARR[@]}"}` is the safe-under-set-u expansion
|
||||||
|
# for arrays that may be empty (CONFIDENTIAL_TERMS is empty
|
||||||
|
# when the operator hasn't bootstrapped ~/.fai-security/).
|
||||||
|
scan_terms_ci "confidential term" "$diff" ${CONFIDENTIAL_TERMS[@]+"${CONFIDENTIAL_TERMS[@]}"}
|
||||||
scan_terms_ci "banned phrase" "$diff" "${BANNED_PHRASES[@]}"
|
scan_terms_ci "banned phrase" "$diff" "${BANNED_PHRASES[@]}"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -208,7 +244,7 @@ scan_message_mode() {
|
||||||
fail "Co-Authored-By: Claude trailer is forbidden (memory: feedback_coding_style.md)"
|
fail "Co-Authored-By: Claude trailer is forbidden (memory: feedback_coding_style.md)"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
scan_terms_ci "confidential term in commit message" "$msg" "${CONFIDENTIAL_TERMS[@]}"
|
scan_terms_ci "confidential term in commit message" "$msg" ${CONFIDENTIAL_TERMS[@]+"${CONFIDENTIAL_TERMS[@]}"}
|
||||||
scan_terms_ci "banned phrase in commit message" "$msg" "${BANNED_PHRASES[@]}"
|
scan_terms_ci "banned phrase in commit message" "$msg" "${BANNED_PHRASES[@]}"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -217,6 +253,16 @@ scan_ci_mode() {
|
||||||
if ! git rev-parse --quiet --verify "$base" >/dev/null 2>&1; then
|
if ! git rev-parse --quiet --verify "$base" >/dev/null 2>&1; then
|
||||||
base=$(git rev-list --max-parents=0 HEAD | tail -1)
|
base=$(git rev-list --max-parents=0 HEAD | tail -1)
|
||||||
fi
|
fi
|
||||||
|
# In CI the confidentiality list is non-optional: a missing
|
||||||
|
# file means an unbootstrapped runner is reviewing changes,
|
||||||
|
# and we'd be silently letting confidential terms through.
|
||||||
|
# The deploy step that wires up the runner is responsible for
|
||||||
|
# placing the file at $FAI_BANNED_TERMS_FILE.
|
||||||
|
if [ "${#CONFIDENTIAL_TERMS[@]}" -eq 0 ]; then
|
||||||
|
fail "CI mode requires \$FAI_BANNED_TERMS_FILE (or ~/.fai-security/banned-terms.txt) to exist"
|
||||||
|
fail " bootstrap the runner with a private confidentiality list before re-running"
|
||||||
|
return
|
||||||
|
fi
|
||||||
local files diff
|
local files diff
|
||||||
files=$(git diff --name-only --diff-filter=ACMR "$base"..HEAD)
|
files=$(git diff --name-only --diff-filter=ACMR "$base"..HEAD)
|
||||||
diff=$(git diff --no-color -U0 "$base"..HEAD -- . "${EXCLUDES[@]}" \
|
diff=$(git diff --no-color -U0 "$base"..HEAD -- . "${EXCLUDES[@]}" \
|
||||||
|
|
|
||||||
|
|
@ -153,9 +153,19 @@ git add tests/server.pem
|
||||||
assert_content_passes "tests/*.pem allowlisted"
|
assert_content_passes "tests/*.pem allowlisted"
|
||||||
|
|
||||||
d=$(setup_repo); cd "$d"
|
d=$(setup_repo); cd "$d"
|
||||||
echo "Notes from the ITDZ Berlin meeting" > confidential.md
|
# The confidentiality list lives outside the repo at
|
||||||
|
# $FAI_BANNED_TERMS_FILE (default ~/.fai-security/banned-terms.txt).
|
||||||
|
# For the test we point at a temp file holding a single
|
||||||
|
# generic test term — the actual production terms stay in the
|
||||||
|
# operator's untracked file and never enter this test suite.
|
||||||
|
TEST_BANNED_TERMS=$(mktemp -t fai_banned.XXXXXX)
|
||||||
|
printf "\\\\bSYNTHETIC_BANNED_TERM_XYZZY\\\\b\n" > "$TEST_BANNED_TERMS"
|
||||||
|
export FAI_BANNED_TERMS_FILE="$TEST_BANNED_TERMS"
|
||||||
|
echo "Document contains SYNTHETIC_BANNED_TERM_XYZZY in passing" > confidential.md
|
||||||
git add confidential.md
|
git add confidential.md
|
||||||
assert_content_fails "confidential term ITDZ" "ITDZ"
|
assert_content_fails "confidential term (synthetic test term)" "SYNTHETIC_BANNED_TERM_XYZZY"
|
||||||
|
rm -f "$TEST_BANNED_TERMS"
|
||||||
|
unset FAI_BANNED_TERMS_FILE
|
||||||
|
|
||||||
d=$(setup_repo); cd "$d"
|
d=$(setup_repo); cd "$d"
|
||||||
echo "Our viral release of a powerful tool that just works" > marketing.md
|
echo "Our viral release of a powerful tool that just works" > marketing.md
|
||||||
|
|
|
||||||
|
|
@ -32,9 +32,14 @@ feel like they're reading a translation.
|
||||||
- Banned words: "viral", "killer", "powerful", "just works",
|
- Banned words: "viral", "killer", "powerful", "just works",
|
||||||
"revolutionary", "game-changing", "seamless", "next-generation",
|
"revolutionary", "game-changing", "seamless", "next-generation",
|
||||||
"cutting-edge", "world-class", "unprecedented".
|
"cutting-edge", "world-class", "unprecedented".
|
||||||
- No private organisation names: never "ITDZ", "Kammergericht",
|
- No private organisation, person, or location names. Pilot
|
||||||
"HTW", "J∆I", or any specific institution. Generic terms only —
|
customers, internal codenames, specific institutions, and
|
||||||
"a regulated organisation", "a public administration".
|
specific cities never appear in Today copy. Generic terms
|
||||||
|
only — "a regulated organisation", "a public administration",
|
||||||
|
"a critical-infrastructure operator". The operator's
|
||||||
|
banned-terms list at ~/.fai-security/banned-terms.txt is the
|
||||||
|
authoritative source; this prompt deliberately doesn't
|
||||||
|
enumerate the strings.
|
||||||
- No emoji.
|
- No emoji.
|
||||||
- No "Co-authored-by Claude" or other AI attribution.
|
- No "Co-authored-by Claude" or other AI attribution.
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue