Microsoft’s Git Fork Equipped to Handle Vast Monorepos

Microsoft has integrated its Git Virtual File System (GVFS) and Scalar optimizations into a customized variant of Git, engineered to support colossal repositories and massive distributed teams.

Essential Designs Team

|

April 22, 2024

MicrosoftUpdate
Software Development
Windows11
LinuxOnWindows
Developer Tools
A grid background

Creating applications on an enormous scale can't compete with the production of an operating system like Windows, especially when it comes down to source code control. The question is, how can one manage the repository (or repositories) of such a mammoth software, with diverse masses of developers, and a complex build pipeline continuously churning out fresh code?

Microsoft's journey with internal source control systems is rather intricate. You may assume it utilised the now obsolete Visual SourceSafe, but that was best suited for local file systems and compact projects. Instead, Microsoft used an array of tools over time, beginning with an internal variant of the habitual Unix Revision Control System, culminating in the standardisation on Perforce Source Depot.

While Visual Studio’s Team Foundation Server was deployed by portions of the company, a shift soon occurred towards using Git as the common engineering platform for the entire firm. This transition was quite logical, given that Git was built to handle the hurdles of managing a humongous code base spread across a slew of globally distributed developers. Similarities between the construction of Windows and Linux illustrate Git’s comprehensive functionality.

However, an issue arises with huge repositories like Windows. Despite their undeniable complexity, tools such as Windows and Office are developed in monolithic repositories that occupy enormous quantities of storage space. The issue results from Git's replication protocol, which copies the entire repository and each modification to every single copy, causing potential overload for developers and the network.

To address this, Microsoft created a method to use Git repositories as a virtual file system, generating local files only when necessary instead of transferring the whole repository through an uncertain network. Hence, the Git Virtual File System (GVFS) was introduced as a tool that carefully aligns Git’s capabilities with Microsoft’s development requirements. While it doesn’t modify Git, it does compromise its offline capabilities.

GVFS is engineered to monitor your workspace and your .git folder, loading only what’s required for your project and checking out only the files necessary. However, you can still view the contents of the repository as an extension of your computer’s file system, much like the way files on OneDrive are accessed.

Going beyond GVFS, Microsoft worked on scaling Git through Scalar. As part of the company's Git fork, this software is designed to assist monorepos. Scalar amplifies any Git repository, no matter its host, but does require Microsoft’s custom Git variant. Its focus is improving Git performance, and it can be utilized independently or in conjunction with GVFS.

Scalar operates in the background, managing registered repositories through a tiered approach to file management. However, it ensures intense repository maintenance operations are handled in the background without obstructing your current task.

Microsoft continues to refine Scalar and has released its own Git distribution with the Scalar CLI. With Microsoft's Git for Windows, macOS, and Linux, users can leverage its "advanced Git features," a concept that aligns with Microsoft's objective to make Git scalable.

Scalar serves as a testing site for features Microsoft hopes to incorporate into Git. By creating its variant of Git, Microsoft can experiment with these features before proposing them to the wider Git community.

Microsoft's approach is reasonable, making the code accessible for community evaluation before any pull requests. With many entities relying on Git, it's vital that any changes do not compromise the functioning for its massive user base and the billions of code lines held in repositories worldwide. While Scalar and GVFS could be essential tools for extensive projects, they won’t interfere with your typical Git usage until officially integrated and then transitioning is as simple as an update on a server.

Share this post

MicrosoftUpdate
Software Development
Windows11
LinuxOnWindows
Developer Tools
Essential Designs logo in black and white

Essential Designs Team

April 22, 2024

A grid background