Rewriting History Safely: filter-repo and Protected Refs

Part 7 of 8 in the Beneath the Porcelain series: what you can build, break, and repair using nothing but git plumbing.

Part 6 ended on a deliberate caveat: a signed tag proves who authorized a release, not that the release is safe. Sometimes it is not, in the most concrete way possible — a secret committed by mistake, sitting in history, retrievable by anyone who can clone the repository, regardless of whether it was ever removed from the current tree. Deleting the file in a new commit does not help; the object is still reachable from an older commit, and git log -p -- path/to/file finds it in seconds.

The actual fix is rewriting history: producing new commits that never contained the secret, and getting every existing clone to adopt that rewritten history instead of the poisoned original. Git provides git filter-repo for the first half, and this series has already built the second half without knowing it — the pre-receive hook from Part 3 extends naturally into a protected-ref rule that makes this kind of rewrite something a server can require to be deliberate, rather than something any force-push can do by accident.

The Leak

A config.env committed, then "fixed" in a later commit that just moves on:

$ git log --oneline
f660560 feat: v2
0decaeb chore: add config
fcc713a feat: initial app

$ git log --oneline --all -p -- config.env
0decaeb chore: add config
diff --git a/config.env b/config.env
new file mode 100644
index 0000000..1bff11e
--- /dev/null
+++ b/config.env
@@ -0,0 +1 @@
+AWS_SECRET_ACCESS_KEY=AKIAABCDEFGHIJKLMNOP

The current tip of main, f660560, does not contain config.env at all if it was deleted in a later commit — but the secret is still sitting in 0decaeb, fully reachable, and anyone who already cloned the repo has it on disk in their own .git directory, whether or not they ever looked at that file.

Removing It With filter-repo

git filter-repo (the modern, actively-maintained replacement for git filter-branch, which the git project itself recommends against for exactly this task) rewrites every commit in history, dropping the specified path from each one:

$ git filter-repo --path config.env --invert-paths
Parsed 3 commits
New history written in 0.04 seconds; now repacking/cleaning...
Repacking your repo and cleaning out old unneeded objects
Completely finished after 0.13 seconds.

--path config.env selects that path; --invert-paths flips the selection from "keep only this" to "remove this and keep everything else." Checking the result:

$ git log --oneline --all
d28951b feat: v2
fcc713a feat: initial app

$ git log --oneline --all -p -- config.env
$ git ls-tree -r HEAD --name-only
app.py

Two things happened, and only one of them is obvious from the commit count. First, config.env's history is gone — the git log -p for that path now returns nothing at all. Second, the commit that existed only to add that file, chore: add config, disappeared entirely rather than becoming an empty commit: filter-repo prunes commits that become no-ops once the filtered path is removed, by default. Three commits became two, not three-with-one-emptied.

It is worth actually confirming the secret is gone from the object store, not just from the visible tree, because that is the property that matters:

$ grep -r "AKIAABCDEFGHIJKLMNOP" .git
$ echo $?
1

No match, exit code 1. filter-repo runs its own gc after rewriting, expiring the reflog and pruning now-unreachable objects, which is why the old blob is not just unreferenced but actually gone from .git rather than lingering as a dangling object waiting to be garbage collected later.

When the File Should Stay and Only the Secret Should Go

--invert-paths is the right tool when the whole file was a mistake. It is the wrong tool when a file has other, legitimate content sitting next to the leaked value — deleting config.env outright would also delete the region and debug flag sitting in the same file:

$ cat app.py.bak
DEBUG=true
AWS_SECRET_ACCESS_KEY=AKIAABCDEFGHIJKLMNOP
REGION=us-east-1

--replace-text takes a file of find==>replace rules and substitutes the match everywhere it appears across every blob in history, leaving the rest of each file untouched:

$ echo "AKIAABCDEFGHIJKLMNOP==>REDACTED" > replacements.txt
$ git filter-repo --replace-text replacements.txt
Parsed 2 commits
New history written in 0.06 seconds; now repacking/cleaning...

$ cat app.py.bak
DEBUG=true
AWS_SECRET_ACCESS_KEY=REDACTED
REGION=us-east-1
$ grep -r "AKIAABCDEFGHIJKLMNOP" .git
$ echo $?
1

The rest of the file is byte-for-byte identical to before; only the literal secret value is gone, from every commit that ever contained it, with the same confirmed absence from the object store as the whole-file removal above. Reach for --invert-paths when an entire file should never have been committed; reach for --replace-text when a file is legitimate and one specific value inside it is not.

Redistributing Without Breaking Every Clone

This is the half of the problem that tutorials on filter-repo tend to skip, and it is the more consequential half in practice: every SHA from the point of the leak forward has changed, because a commit's hash depends on its content and its parent's hash, and the rewrite altered both for 0decaeb and everything built on top of it. Any clone still tracking the old history is now looking at commits your server no longer has, and a plain git pull from that clone does not know how to reconcile the two:

$ git pull real-origin main
hint: You have divergent branches and need to specify how to reconcile them.
fatal: Need to specify how to reconcile divergent branches.

That is git correctly refusing to guess. A merge or rebase here would try to reconcile two histories that, from git's perspective, share no common ancestor after the rewrite point — the old f660560 and the new d28951b have entirely different SHAs despite representing the same final app.py content. The correct move for a downstream clone is not to merge; it is to discard the poisoned history outright and adopt the clean one:

$ git fetch real-origin
+ f660560...d28951b main -> real-origin/main  (forced update)
$ git reset --hard real-origin/main
HEAD is now at d28951b feat: v2

git fetch itself already flags this as a "forced update" on the remote-tracking ref — that phrase in fetch output is your signal that this is exactly the divergent-history situation, before you even try to reconcile anything. reset --hard throws away the local branch's history and points it at the new tip; any local, unpushed commits made on top of the old history need to be individually cherry-picked onto the new base by hand, because there is no automatic way to carry them across a history this discontinuous.

Making the Rewrite Deliberate: A Protected-Ref Rule

Redistributing a rewritten history means force-pushing it — by definition, the new tip is not a descendant of the old one. That is exactly the shape of push a well-behaved server should be suspicious of by default, and exactly the shape Part 3's pre-receive hook already knows how to inspect. Extending it to require an explicit, intentional override for non-fast-forward pushes to main turns "someone force-pushed and blew away history" from a silent possibility into an action that has to be requested by name:

#!/usr/bin/env bash
set -euo pipefail
zero="0000000000000000000000000000000000000000"
 
allow_rewrite=0
for i in $(seq 0 $((${GIT_PUSH_OPTION_COUNT:-0} - 1))); do
  opt_var="GIT_PUSH_OPTION_$i"
  if [ "${!opt_var:-}" = "allow-history-rewrite" ]; then
    allow_rewrite=1
  fi
done
 
while read -r oldrev newrev refname; do
  [ "$refname" = "refs/heads/main" ] || continue
  [ "$oldrev" = "$zero" ] && continue
 
  if git merge-base --is-ancestor "$oldrev" "$newrev"; then
    continue   # fast-forward, always fine
  fi
 
  if [ "$allow_rewrite" = "1" ]; then
    echo "pre-receive: non-fast-forward on $refname allowed via -o allow-history-rewrite" >&2
    continue
  fi
 
  echo "REJECTED: $refname is protected against non-fast-forward pushes." >&2
  echo "If this is a deliberate history rewrite, push with: git push -o allow-history-rewrite" >&2
  exit 1
done

GIT_PUSH_OPTION_COUNT and GIT_PUSH_OPTION_0, _1, and so on are environment variables git populates from --push-option (-o) flags on the client's git push — but only for pre-receive and post-receive; the older, per-ref update hook does not receive them at all, which is worth knowing before you spend time debugging why an update-hook version of this check never sees the flag. The server also has to opt in with receive.advertisePushOptions true, or the client's -o flags are dropped before they ever reach the wire.

git merge-base --is-ancestor "$oldrev" "$newrev" is the actual fast-forward test: it succeeds silently if oldrev is an ancestor of newrev, which is precisely the definition of a fast-forward. An ordinary push passes through untouched:

$ git push --force origin main
 ! [rejected]        main -> main (fetch first)

(Git's own local fast-forward check catches an ordinary divergence before the hook even runs — --force is required just to get the push attempted at all, since the client already suspects a conflict.) The rewritten history, pushed without the override, gets caught server-side even with --force:

$ git push --force origin main
remote: REJECTED: refs/heads/main is protected against non-fast-forward pushes.
remote: If this is a deliberate history rewrite, push with: git push -o allow-history-rewrite
 ! [remote rejected] main -> main (hook declined)

And with the explicit override, it goes through, with the hook logging that the override was actually used:

$ git push --force -o allow-history-rewrite origin main
remote: pre-receive: non-fast-forward on refs/heads/main allowed via -o allow-history-rewrite
 + f660560...d28951b main -> main (forced update)

The value here is not that the rewrite becomes hard to do — an administrator with server access could always force it through some other path. The value is that it stops being something a routine push --force can trigger by accident, and starts being something that shows up explicitly, by name, in server logs and in the command the person pushing had to type.

Removing the Secret From History Is Not the Same As Revoking It

Everything in this part operates on git history, and it is worth being direct about what it does not do: none of it revokes the leaked AWS key itself. The moment AKIAABCDEFGHIJKLMNOP was pushed to any remote — even briefly, even if the very next commit "fixed" it — it should be treated as compromised, full stop, regardless of whether it ever gets rewritten out of history at all. Anyone who cloned the repository in that window has it on disk, in a location no filter-repo invocation anywhere will ever reach, because it was never git's object database that mattered at that point; it was the value itself, now sitting in an arbitrary number of .git directories this series has no visibility into.

The actual incident-response order is: rotate the credential at the provider first, confirm the old one no longer works, and only then spend time on the history rewrite. The rewrite is about hygiene, audit cleanliness, and stopping a git log -p from casually surfacing an old, revoked value that still looks alarming to whoever finds it next — real reasons to do it, but secondary ones. Treating the rewrite as the fix, rather than the follow-up, is the most common mistake in this exact scenario, and it is worth naming plainly rather than letting the mechanics of filter-repo imply otherwise.

Takeaways

filter-repo rewrites objects, not just the tree at HEAD, and prunes commits that become empty as a result. Confirm a leak is actually gone by grepping .git directly, not by checking whether the current tree still contains the file — those are different questions, and only the first one is the one that matters for a leaked secret.

--invert-paths removes a whole file from history; --replace-text removes one value while leaving everything else intact. Choosing between them is a question about the file, not the secret: if the surrounding content is legitimate, redact the value in place rather than deleting a file that has every right to still exist.

Rewriting history is cleanup, not incident response. Rotating the leaked credential at its source, before it is treated as anything other than compromised, is the step that actually matters; the filter-repo pass that follows is about not leaving an alarming, if already-revoked, value sitting in git log -p for the next person who goes looking.

A rewritten history is, to every existing clone, indistinguishable from a completely unrelated one past the rewrite point. git fetch reporting a "forced update" on your tracking ref is the signal to reset --hard, not to merge or rebase — and any local work built on the old history has to be manually replayed on top of the new one.

Fast-forward-only-by-default plus an explicit, named override is the general pattern for making a dangerous operation safe without making it impossible. The same pre-receive mechanism from Part 3 that ran tests against incoming code can just as easily inspect the shape of a ref update, and require the one class of push that should never happen by accident to say so out loud.

Next: Part 8: A Deployment Pipeline Made Entirely of Git — the capstone. Combining the bare repo, the hooks, the blue-green worktrees, and the signed release gate into one system, and being honest about where it stops being the right choice.

Beneath the Porcelain is an eight-part series by Ajitem Sahasrabuddhe on the git internals that quietly hold together small production systems. Every command shown was run against git 2.50.1 on macOS before being written up; output may differ slightly on other platforms or versions.