Git 2.55 for large repositories: how incremental MIDX lowers maintenance cost

When the problem is not code, but repository history

The team has already made tests faster, cached dependencies, and removed unnecessary build steps, yet CI still begins the day with several slow Git operations. fetch takes longer than expected, checkout sometimes feels like its own job, and nightly maintenance consumes runner time. In these cases, the cause is often not new application code. It is the way repository history has accumulated over years.

Git 2.55 is interesting for exactly this kind of large project. The release adds a way to run git repack --write-midx=incremental, and to combine it with --geometric=2 -d. This is not a magic switch for every repo, but it is a useful tool for teams that already feel the cost of storing history: many packfiles, slow service operations, heavy mirrors, and CI runners that spend real time preparing the repository before the actual build starts.

A quick map of Git storage

To understand the change, imagine .git/objects not as a mysterious service folder, but as an archive. Git object records can start as separate files, but over time Git places them into a packfile. That saves disk space and improves read performance because Git can work with larger organized blocks instead of thousands of tiny files.

Next to packfiles, Git keeps indexes so it can find the object it needs quickly. When there are many packfiles, a new question appears: how does Git know which packfile contains the requested record? This is where MIDX, or multi-pack-index, helps. It provides one lookup layer across several packfiles and reduces unnecessary searching during reads.

A classic MIDX is simple to read, but it has an uncomfortable maintenance cost in large repositories. If one index describes a huge set of packfiles, even a small update can require rewriting a large metadata file. That is painful during regular maintenance, where the team does not want to touch the whole archive just to account for a few new blocks.

What incremental MIDX adds in Git 2.55

Incremental MIDX stores the index as a chain of layers instead of one large file. Older layers can stay in place, while a new layer is appended for new packfiles. The idea is simple: if a repo receives new commits every day, Git should not always have to rewrite index data for the whole object store.

In Git 2.55, git repack can write these incremental MIDX chains directly:

git repack --write-midx=incremental

For real maintenance, the second part matters: the layer chain must not grow forever. Git 2.55 therefore supports combining incremental MIDX with geometric repacking:

git repack --write-midx=incremental --geometric=2 -d

The practical meaning is that newer, smaller layers can be rewritten more often, while older and larger layers do not need to be touched on every run. The GitHub Blog describes this as a compromise between a single large MIDX, which is efficient to read but expensive to update, and an endlessly appended chain of small layers, which is cheap to write but eventually needs maintenance of its own.

A lab test on a copy

Start with a separate copy of the large repository, not with a global CI change. A minimal baseline looks like this:

git --version
du -sh .git/objects
find .git/objects/pack -name '*.pack' | wc -l
time git fetch --all --prune

If you already have a maintenance command, measure that too. For example, record how long the current git gc, git repack, or git maintenance run takes. Then run the new mode on the test copy:

time git repack --write-midx=incremental --geometric=2 -d

After the command finishes, repeat the same measurements. Do not look only for one impressive run. Watch the behavior across several cycles: add new objects, run fetch, run maintenance again, then compare time and packfile count. If the first run is expensive but later runs become noticeably cheaper, that is a useful signal. If nothing changes, your repo may be too small, or the real bottleneck may be somewhere else: network, dependency caches, working tree size, or CI configuration.

Rollout without heroics

A good rollout for this kind of change is boring. First, use one test runner or mirror. Next, try one non-critical repo with large history. Then compare metrics for several days. Only after that should the team consider broader adoption.

The rollback plan should say clearly how to return to the previous maintenance process. Before the experiment, save the current commands, Git version, job schedule, and baseline metrics. If the team uses shared caches or mirrors, do not change them without a backup. A mistake in Git’s service storage can block not one build, but the whole CI queue.

Also, do not draw conclusions from a small repo. Incremental MIDX is built for situations where the number of objects and packfiles has become an operational cost. On a small project, the new mode may show no visible difference, and that is fine.

Anti-patterns

The first anti-pattern is running the new repack on a production mirror without a copy. Even though the command is official, maintenance changes the repository’s service structure, not just a nice statistic on a dashboard.

The second is measuring only .git size after one run. For this topic, repeated maintenance time, packfile count, result stability, and the effect on fetch or checkout matter more.

The third is enabling the change for every repository in the same way. A large monorepo and a small service have different problems. For one, incremental MIDX may lower maintenance cost. For the other, it may add complexity with no visible benefit.

Conclusion

Git 2.55 does not remove the need for repository discipline, but it gives platform teams another practical lever. If a large repo already spends minutes on Git operations, test incremental MIDX in a lab, collect metrics, and roll it out in small steps. The best outcome here is not a loud release announcement. It is a calmer CI system that stops paying unnecessary cost for years of history.

Sources

• GitHub Blog: Highlights from Git 2.55 — https://github.blog/open-source/git/highlights-from-git-2-55/
• Git documentation: git-repack — https://git-scm.com/docs/git-repack
• Git documentation: git-multi-pack-index — https://git-scm.com/docs/git-multi-pack-index