Custom Merge and Filter Drivers for Generated Files

Part 5 of 8 in the Beneath the Porcelain series: what you can build, break, and repair using nothing but git plumbing.

Every pipeline in this series so far has treated files as opaque blobs: check them out, checksum them, rsync them. Git's default merge strategy treats them almost as bluntly — a three-way text diff, line by line, with no idea what the file actually means. That is a fine default for source code. It is a bad default for the generated files that inevitably end up in a repository anyway: lockfiles, compiled config, build manifests. Two branches that both add an unrelated dependency to the same lockfile do not have a real disagreement, but a line-based diff cannot tell the difference between "these two changes are compatible" and "these two changes are opposed," so it reports a conflict either way.

Git has had the fix for this since long before it was common knowledge: .gitattributes can assign a merge driver to specific paths, and that driver — built-in or entirely custom — decides how a three-way merge resolves for files matching that path, instead of the generic line-based algorithm. A related but separate mechanism, clean/smudge filters, controls what actually gets stored in a blob versus what appears in your working tree, which solves a different but related problem: generated files whose content is deterministic except for one noisy field, like a build timestamp, that turns every rebuild into a diff.

The Conflict That Should Not Be a Conflict

A lockfile-shaped file, one dependency per line, with two branches each adding a different, unrelated line:

$ cat deps.lock
lodash==4.17.21
react==18.2.0

Branch feature-a adds axios; branch feature-b, cut from the same base, adds zod. Merging the first into main is uneventful. Merging the second is not:

$ git merge feature-b
Auto-merging deps.lock
CONFLICT (content): Merge conflict in deps.lock
Automatic merge failed; fix conflicts and then commit the result.

Git's default merge driver does not know that deps.lock is a set of independent entries where order barely matters. It sees two branches that both edited "the same region" of the file — the end of it — and reports a conflict, even though the actual intent of both changes is obviously compatible to any human glancing at it.

Git ships a built-in driver for exactly this shape of file, and it is one line in .gitattributes away:

$ git merge --abort
$ echo "deps.lock merge=union" > .gitattributes
$ git add .gitattributes && git commit -m "chore: union-merge the lockfile"
$ git merge feature-b
Auto-merging deps.lock
Merge made by the 'ort' strategy.

$ cat deps.lock
lodash==4.17.21
react==18.2.0
axios==1.6.0
zod==3.22.0

merge=union tells git to take the union of lines across both sides instead of diffing them positionally, deduplicating anything identical on both sides. It is a real, built-in strategy (git help gitattributes documents it under merge), configured entirely through .gitattributes — no script, no git config entry, and it applies automatically to anyone who clones the repo with that .gitattributes file present, including CI.

Union merge is well-suited to line-oriented files, and badly suited to anything structured, because it has no concept of "these two lines both set the same key to different values" — it would just keep both lines and produce invalid output. For that, you need an actual custom driver.

A Real Custom Driver: Per-Key JSON Merge

manifest.json is a generated map from package name to version, and it deserves a merge strategy that understands it is a map, not text:

{
  "lodash": "4.17.21",
  "react": "18.2.0"
}

A custom merge driver is a script invoked as <driver> %O %A %B: paths to temp files holding the merge base, "ours," and "theirs," with the expectation that the script writes its resolution back into the %A path and exits 0 for a clean merge or non-zero to signal a real conflict (at which point git falls back to leaving conflict markers, same as it would without any driver at all).

#!/usr/bin/env python3
# merge-manifest.py %O %A %B
import json, sys
 
base_path, ours_path, theirs_path = sys.argv[1:4]
 
def load(path):
    try:
        with open(path) as f:
            return json.load(f)
    except (json.JSONDecodeError, FileNotFoundError):
        return {}
 
base, ours, theirs = load(base_path), load(ours_path), load(theirs_path)
keys = set(base) | set(ours) | set(theirs)
merged, conflicts = {}, []
 
for key in keys:
    b, o, t = base.get(key), ours.get(key), theirs.get(key)
    if o == t:
        if o is not None:
            merged[key] = o
    elif o == b:
        if t is not None:
            merged[key] = t
    elif t == b:
        if o is not None:
            merged[key] = o
    else:
        conflicts.append(key)
        merged[key] = t
 
with open(ours_path, "w") as f:
    json.dump(merged, f, indent=2, sort_keys=True)
    f.write("\n")
 
if conflicts:
    sys.stderr.write(f"merge-manifest: real conflicts on keys: {conflicts}\n")
    sys.exit(1)
sys.exit(0)

The core logic is an actual per-key three-way merge: if both sides agree, keep it; if only one side changed a key relative to the base, take the side that changed; if both sides changed the same key to different values, that is a genuine conflict, and the script says so on stderr rather than silently picking a winner.

Wiring it in takes a driver name in .gitattributes and a git config entry pointing at the script:

$ git config merge.manifest.name "per-key JSON merge for manifest.json"
$ git config merge.manifest.driver "python3 .git-drivers/merge-manifest.py %O %A %B"
$ echo "manifest.json merge=manifest" >> .gitattributes

Two branches adding different keys — feature-c adds axios, feature-d adds zod — now merge without incident:

$ git merge feature-d
Auto-merging manifest.json
Merge made by the 'ort' strategy.

$ cat manifest.json
{
  "axios": "1.6.0",
  "lodash": "4.17.21",
  "react": "18.2.0",
  "zod": "3.22.0"
}

And when both branches genuinely do change the same key to different values — feature-e bumps react to 18.3.0, feature-f bumps it to 19.0.0 — the driver correctly refuses to guess:

$ git merge feature-f
merge-manifest: real conflicts on keys: ['react']
Auto-merging manifest.json
CONFLICT (content): Merge conflict in manifest.json
Automatic merge failed; fix conflicts and then commit the result.

$ git status --short
UU manifest.json

This is exactly the behavior you want from a merge driver: silent, automatic resolution for the changes that are genuinely independent, and an honest conflict — not a coin flip — for the one key both branches actually disagree about. git status reports it the same way it reports any unmerged file, so the rest of your conflict-resolution workflow does not need to know a custom driver was ever involved.

A Lighter-Weight Tool: textconv for Readable Diffs

Not every generated file needs a merge driver. Sometimes the actual problem is smaller: a minified or otherwise single-line file whose diffs are unreadable, even when the underlying change is trivial. A minified JSON bundle stored on one line diffs as a single opaque line replacement, no matter how small the real change is:

$ cat bundle.min.json
{"a":1,"b":2}
$ echo '{"a":1,"b":2,"c":3}' > bundle.min.json   # (minified, one line)
$ git diff -- bundle.min.json
-{"a":1,"b":2}
+{"a":1,"b":2,"c":3}

textconv runs a conversion command on a file's content purely for the purpose of generating a diff — it never touches what git actually stores, unlike clean/smudge below. Registering one for minified JSON is a git config entry plus a matching .gitattributes line, using the same diff= attribute the union merge driver's cousin (diff= rather than merge=) hooks into:

$ git config diff.prettyjson.textconv "python3 -m json.tool"
$ echo "bundle.min.json diff=prettyjson" >> .gitattributes

The same one-line change now diffs as a readable, line-oriented pretty-print, with git diff running the file through python3 -m json.tool on both sides before comparing:

$ git diff -- bundle.min.json
 {
     "a": 1,
-    "b": 2
+    "b": 2,
+    "c": 3
 }

textconv is worth reaching for before a full merge driver whenever the actual pain point is "I cannot read this diff," rather than "this file produces false conflicts." It is a smaller commitment — one git config line, no script beyond a formatter that likely already exists — for a problem that is genuinely more common than the false-conflict case the rest of this part focuses on.

Clean and Smudge: Filtering What Git Actually Stores

Merge drivers solve "these two changes shouldn't conflict." Clean/smudge filters solve a related but different problem: a generated file that is byte-for-byte identical between two builds except for one field that always changes, like a timestamp, which turns every rebuild into a spurious diff or an unnecessary merge conflict of its own.

A clean filter runs on a file's content on its way into the object database — at git add or git commit time. A smudge filter runs in the opposite direction, on the way out, when git writes a blob into your working tree at checkout. The two together mean git can store one canonical form while your working tree sees another.

#!/usr/bin/env python3
# stamp-clean.py: strip the volatile field before it becomes a blob.
import json, sys
data = json.load(sys.stdin)
data["generatedAt"] = "STABLE"
json.dump(data, sys.stdout, indent=2, sort_keys=True)
sys.stdout.write("\n")

#!/usr/bin/env python3
# stamp-smudge.py: re-inject a real timestamp on checkout.
import json, sys, datetime
data = json.load(sys.stdin)
data["generatedAt"] = datetime.datetime.now(datetime.timezone.utc).isoformat()
json.dump(data, sys.stdout, indent=2, sort_keys=True)
sys.stdout.write("\n")

$ git config filter.stamp.clean "python3 .git-drivers/stamp-clean.py"
$ git config filter.stamp.smudge "python3 .git-drivers/stamp-smudge.py"
$ echo "version.json filter=stamp" >> .gitattributes

Build the file with a real timestamp, and confirm what git actually stored is the stable version:

$ cat version.json
{
  "build": 42,
  "generatedAt": "2026-08-06T14:49:44.356412"
}
$ git add version.json
$ git show :version.json
{
  "build": 42,
  "generatedAt": "STABLE"
}

The working tree has today's real timestamp; the staged blob has the placeholder. Commit it, then simulate a second build a moment later with a new timestamp but the same build number:

$ git commit -m "build: version.json #1"
$ python3 rebuild.py   # writes a fresh timestamp, same build=42
$ git diff --stat
$ git status --short

Both come back empty. Git applies the clean filter before comparing the working tree to the index, so a file that differs only in a field the filter strips is correctly seen as unchanged — no diff, no accidental commit, no merge conflict on a field nobody actually cares about. Check the file out fresh, and smudge reintroduces a genuine timestamp into the working copy:

$ git checkout -- version.json
$ cat version.json
{
  "build": 42,
  "generatedAt": "2026-08-06T14:49:56.524240"
}

Git only ever stored "STABLE". The real timestamp exists solely in your working tree, regenerated by smudge every time the file is checked out.

It is worth being precise about what this does and does not protect against. The clean filter only ever runs on content on its way into a git object; if two branches both modify version.json in a way that touches the build field itself, not just generatedAt, that is a real change on both sides and merges (or conflicts) exactly as any other line-based file would, filter or no filter. clean/smudge neutralizes one specific volatile field; it does not turn the file into something a merge driver would treat structurally, the way manifest.json's per-key driver does above. The two mechanisms are complementary, not interchangeable, and a file that needs both — volatile fields stripped and structural per-key merging — can carry a merge= attribute and a filter= attribute at the same time, in the same .gitattributes line, each doing its own independent job.

Takeaways

Merge drivers change how a conflict is decided; clean/smudge filters change what a blob actually contains. They solve adjacent but distinct problems, and generated files often need both: a merge driver so structurally-independent changes do not spuriously conflict, and a filter so volatile fields do not turn every rebuild into a diff.

A custom merge driver should say no when it should say no. The per-key JSON driver above happily resolves independent changes and explicitly refuses to guess when two branches genuinely disagree, exiting non-zero and letting git fall back to normal conflict markers. A driver that silently picks a side on a real conflict is worse than no driver at all — it is a subtly-wrong merge that will not show up as an error.

textconv is the right first reach for an unreadable diff; a merge driver is the right reach for a false conflict. They solve different problems at different levels of commitment — one config line and an existing formatter versus a script that has to get 3-way merge semantics right — and it is worth trying the cheaper one before reaching for the more powerful one.

.gitattributes makes both mechanisms travel with the repository. Unlike a personal git config alias, a merge=, diff=, or filter= attribute committed into .gitattributes applies to anyone who clones the repo — including CI runners — the moment the corresponding driver is registered in their local git config. The registration step is the one thing that does not travel automatically, which is worth documenting in a setup script if more than one person needs it.

Next: Part 6: Signed Tags as a Release Gate. Using a cryptographic signature, not a branch name, to decide what is actually allowed to reach production.

Beneath the Porcelain is an eight-part series by Ajitem Sahasrabuddhe on the git internals that quietly hold together small production systems. Every command shown was run against git 2.50.1 on macOS before being written up; output may differ slightly on other platforms or versions.