Linuxsecurity

Vault secret store + Git history scrub

Set up a pointers-not-values secret pattern with a pre-commit guard, and purge a secret that was already committed from all Git history with git-filter-repo.

Distrosdebian, ubuntu, opensuse
Shellbash
Updated
Script
bash
# ─────────────────────────────────────────────────────────────
# PART A — Prevention: secret store + pre-commit guard
# ─────────────────────────────────────────────────────────────

# 1. Create a root-only secret store OUTSIDE any Git repo
sudo tee /root/.myofficelab-secrets > /dev/null <<'EOF'
# Values live here. Notes reference the variable NAME only.
PROXMOX_API_TOKEN_ID=youruser@pve!app-token
PROXMOX_API_TOKEN_SECRET=CHANGE_ME
GRAFANA_API_KEY=CHANGE_ME
EOF
sudo chmod 600 /root/.myofficelab-secrets

# 2. In a note, record ONLY a pointer — never the value:
#    "Token: PROXMOX_API_TOKEN_ID/SECRET in /root/.myofficelab-secrets"

# 3. Add a pre-commit hook that blocks secret-shaped staged content
cat > .git/hooks/pre-commit <<'EOF'
#!/bin/bash
if git diff --cached -U0 | grep -nEi \
  '(api[_-]?key|token|secret|password|passwd|BEGIN [A-Z ]*PRIVATE KEY)[[:space:]]*[:=]' \
  | grep -v 'CHANGE_ME'; then
  echo "Possible secret in staged changes — commit blocked."
  echo "Store the value in your secret store; commit only a variable-name pointer."
  exit 1
fi
EOF
chmod +x .git/hooks/pre-commit

# 4. (Recommended) install a real scanner for deeper coverage
#    https://github.com/gitleaks/gitleaks
#    gitleaks detect --source .

# ─────────────────────────────────────────────────────────────
# PART B — Recovery: a secret is already in history
# ─────────────────────────────────────────────────────────────

# 5. ROTATE FIRST. Invalidate the exposed credential at its source
#    (regenerate the token / change the password) BEFORE touching Git.
#    Assume it has already leaked.

# 6. Install git-filter-repo
sudo apt-get install -y git-filter-repo   # or: pipx install git-filter-repo

# 7. Work on a FRESH clone (filter-repo refuses a dirty/linked clone)
git clone /wiki-data/git /tmp/vault-scrub
cd /tmp/vault-scrub

# 8a. Replace a leaked VALUE across every commit
echo 'the-leaked-secret-value==>REDACTED' > /tmp/replacements.txt
git filter-repo --replace-text /tmp/replacements.txt

# 8b. OR remove a whole file that should never have been tracked
# git filter-repo --path secrets.env --invert-paths

# 9. Re-point the origin (filter-repo drops remotes by design) and force-push
git remote add origin /wiki-data/git
git push --force --all
git push --force --tags

# 10. Re-clone EVERY other copy (laptop mount, mirrors) from the rewritten repo.
#     Old clones carry the secret in their history — discard them.

What this does

Two parts. Part A sets up the prevention pattern: a root-only secret store outside Git, notes that reference variable names instead of values, and a pre-commit hook that blocks obviously secret-shaped content. Part B is the recovery path if a secret is already committed — rotate the credential, then purge it from all Git history with git-filter-repo and re-clone every copy.

Why this approach

A knowledge base full of notes about your lab is a very tempting place to store configuration details. IP addresses, usernames, API tokens, passwords — all the stuff you need to remember lives right there alongside your runbooks. And for most of it, that’s fine. The line is between referring to a credential and storing it in the vault. Your runbook can say “the Grafana API key is in GRAFANA_API_KEY in the secret store.” It cannot contain the actual key value.

The reason is Git’s append-only history. Once a value is committed, it exists in every clone, every backup, and every push forever, even if you delete it in a follow-up commit. A “delete” in Git just adds a new commit that removes the file from the working tree — the old commit still has the full value. The only way to actually remove a secret from a Git repo is to rewrite history with a tool like git-filter-repo, and that rewrites every commit hash, invalidates every clone, and requires a force-push. It’s painful every time. The prevention pattern in Part A of this playbook is vastly cheaper than the recovery path in Part B.

The prevention pattern has three parts working together. First, the actual secret values live in a root-owned file outside the Git repo (/root/.myofficelab-secrets, chmod 600) — never in any file that Git can see. Second, notes reference credentials by variable name only: “the API token is $GRAFANA_API_KEY” rather than the value. Third, a pre-commit hook scans staged content for patterns that look like secrets (long hex strings, tokens matching known patterns) and blocks the commit if it finds any. The hook isn’t foolproof — you can always commit something the pattern doesn’t catch — but it stops the most common accidents.

The recovery path exists for when prevention fails. The most important step is the first one: rotate the credential before you do anything else. A secret that has been committed to Git history should be treated as compromised regardless of whether the repo is public or private — backups, mirrors, and anyone who pulled the repo before you rewrote history all have a copy. Rotation makes the old value useless even if it leaks again. Then you scrub history, force-push, and re-clone every copy of the repo. That last step is the one people skip, and it’s the one that leaves a ticking time bomb in their existing SSHFS mount.

Prerequisites

  • A Git vault (see the companion “Stand up a Git + Markdown knowledge vault” playbook).
  • Root/sudo to create /root/.myofficelab-secrets and set chmod 600.
  • For recovery: git-filter-repo installed, and the ability to force-push and re-clone every copy of the repo.

Notes

  • Substitute your own values before running any line — this playbook uses example stand-ins. Replace /root/.myofficelab-secrets (your secret-store path), the PROXMOX_API_TOKEN…/GRAFANA_API_KEY names (your own variable names), and CHANGE_ME / the-leaked-secret-value (your real values). Never commit a real secret into the example.
  • Deleting a secret in a new commit does NOT remove it — Git retains every past version. Only a history rewrite purges it. This is the single most important point.
  • Rotate before you scrub. Once a credential has touched Git history, treat it as compromised regardless of whether the repo is “private”. Rotation is the real fix; the scrub just stops re-leaking the old value.
  • git filter-repo requires a fresh clone and intentionally removes configured remotes as a safety measure — re-add origin before pushing (step 9).
  • A force-push rewrites shared history. Every existing clone and SSHFS-mounted copy diverges and must be re-cloned. Coordinate this if anyone else pulls the repo.
  • CHANGE_ME is deliberately whitelisted by the pre-commit hook so placeholder templates don’t trip it — replace it with the real value only in the root-only store, never in a note.
  • Mark tested: true only after running the recovery path against a throwaway repo first.
All playbooks