Ideas for building on Git's concepts and capabilities
This month, Git is celebrating its 20th anniversary. Over the past two decades, this version control system has become the de facto standard for software version management and development. Furthermore, Git forms the foundation of major platforms such as GitHub and GitLab, supporting tens of millions of users from all over the world. Currently 95 percent of all developers use Git as their primary version control system.
After taking a brief look at the origins of Git and what has been accomplished in software version management over the past 20 years, this article highlights several adjacent domains where Git's versioning concepts can be or already have been successfully applied. In addition we present various ideas for extending Git's capabilities and usefulness in significant ways.
Off to a quick start
Git was created by Linus Torvalds in April 2005 to support the development of the Linux kernel after BitKeeper was no longer (freely) available and no other existing free system offered similar functionality. At the time, Linus had only four design criteria: patching should take no more than three seconds; take CVS as an example of what not to do; support a distributed, BitKeeper-like workflow; and provide very strong safeguards against corruption, whether accidental or malicious.
The newly created system was self-hosting within a week, and the first kernel commits and merges took place within two weeks. By the end of July, the maintenance and further development of Git were transferred to major contributor Junio Hamano, who holds this role to this day.
Git
Git is a distributed version control system mostly used for – but not limited to – the development and documentation of software code, datasets and specifications. Its main feature is the support of massive non-linear development, allowing large numbers of collaborators to participate in the development and maintenance of a project in a semi-independent way.
Projects are brought under Git control locally by creating a set of metadata in a hidden directory. Others can get their own local copies of the very same project through a Git server and work on these copies at their own discretion. Being very precise about the identification of files and patches, and keeping close track of development history, Git allows for fast and easy synchronization of changes between repositories, at the same time providing a high level of granularity in doing so.
In practice, projects are hosted at a central location, from where copies are made by developers and where approved changes are fed back into the main branch. Being fast and highly scalable, Git forms the foundation of major hosting platforms such as GitHub and GitLab, supporting tens of millions of users all over the world.
World domination
I really never wanted to do source control management at all and felt that it was just about the least interesting thing in the computing world
I really never wanted to do source control management at all and felt that it was just about the least interesting thing in the computing world.
That's what Linus said in an interview on the occasion of Git's 10th anniversary. [1] According to him, the biggest problem with BitKeeper was that is wasn't open source, which made a lot of people involved in Linux kernel development not want to use it. After discussions came to a clash with the owner of BitKeeper, it took Linus only a few days to write the basics of Git, as he had already been thinking about its requirements and concepts for some time.
Since its inception 20 years ago, Git has taken over the world of software version management almost completely – much to the surprise of Linus himself. 95 percent of all developers currently use Git as their primary version control system,Stack Overflow Developer Survey 2022:
https://survey.stackoverflow.co/2022/#section-version-control-version-control-systems making it the de facto standard. Git also forms the foundation of major hosting platforms such as GitHub and GitLab, supporting tens of millions of users worldwide in what Linus calls "social coding". He explains the immense success of Git through its "distributed" nature and that it's so easy to start a new project. While Linus used to joke about aiming for world domination with the Linux kernel [1, 2], it appears that it is Git that has now actually achieved this status.
FAIR
FAIR is a set of Guiding Principles to unlock research data and make them machine-actionable:
- Findability: machine-readable and searchable metadata allow for automatic discovery
- Accessibility: (meta)data is accessible through an open protocol that supports authentication and authorization
- Interoperability: unambiguous (meta)data definitions (e.g. ontology) allow data to be integrated with other data, and be used by applications and workflows
- Reusability: data should have clear descriptions, licenses and provenance, and adhere to domain-specific community standards (relevance)
As such, FAIR is closely related to the Semantic Web, which aims to make internet data machine-actionable by adding ontological metadata to allow machine reasoning. FAIR data is also closely related to Open Data, Open Science and FOSS, although openness is not a requirement of FAIR itself.
FAIR was formally defined in 2016 by a consortium of scientific, industrial and other stakeholders [1]. Since then, its principles have been adopted by several research institutes and are actively promoted and researched by all major umbrella organizations in the research-data ecosystem.
The GO FAIR Initiative is an international network aiming at helping implement the FAIR data principles, for example through its GO FAIR International Support and Coordination Offices (GFISCOs) and through the European Open Science Cloud (EOSC).
Free and Open-Source Software
Free and Open-Source Software (FOSS) is characterized by the use of specific copyright licensing. These licenses, however, are not primarily about the copyright of the code produced: in this case copyright legislation is merely used to facilitate collaborative development of public code within a community of (anonymous) contributors.
The current FOSS landscape is largely covered by two main licensing types. Both allow users to run, study, change and (re)distribute the (source) code.
- copyleft: Crucial in these licenses is the requirement that the same rights must be preserved in derivative works. Most importantly this means that others cannot modify the code and redistribute it without also making the modified source code available.
- permissive: These licenses allow but do not require others to make modified source code of derivative works available, while at the same time forbidding any limitations on the usage and further development of the software. This way making sure the original software remains available and can be used with minimal restrictions – even in closed-source commercial software – makes permissive licenses so attractive to businesses and industry.
FOSS licenses form the foundation of a highly dynamic ecosystem, combining strong competition and massive collaborative development and reuse in an evolutionary process. Over the past decades, enormous amounts of FOSS software have been created this way, including Firefox, LibreOffice, Linux, Python and WordPress. Its economic value is measured in tens of billions of euros annually [1].
Law as code
It turns out that if you say "patch" instead of "amendment" and "code freeze" instead of "plenary vote", many members of the software community suddenly understand what you are talking about!EPFSUG: The hacker perspective on lawmaking
Just as software code is a formalization of an algorithm or intent, legislative texts aim to formalize legal code. And just like the development of software, the legislative process requires precise version management and collaboration among many participants.
People involved in legislative processes have recognized these similarities and have put to work 'software version control'-like systems – or even Git itself – for their own purposes.
European Legislation
One example is LEOS, short for Legislation Editing Open Software. [1, 2] This tool facilitates the drafting of legislative texts and generates legislation in an XML format, that way supporting interoperability between European institutions.
LEOS is the best-known FOSS tool created by the European Commission and freely available under the EUPL license. The software is also used by several Member States and various other public administrations.
Parltrack [1] is unrelated but can be considered complementary to LEOS. It combines information on dossiers, representatives, vote results and committee agendas of the European Parliament into a single database and allows the tracking of dossiers using e-mail and RSS. The platform improves the transparency of legislative processes. For example, it allows you to see who are the most influential Members of the European Parliament related to a specific dossier.
Most of the data presented on the website is also available in JSON format for further processing, just as a dump of the full database. The Parltrack software itself is available under a free software license.
"GitLaw"
Both LEOS and Parltrack attach great importance to the history of the development process, one of the strongpoints of Git. Washington D.C. has even published an authoritative copy of its laws on GitHub [1]. It allowed one of its citizens to fix a typo using a pull request [1].
Others have been philosophizing about a public "GitLaw" system specifically for legislative texts. Such a system would allow both drafters and citizens (through crowdsourcing) to propose bills and amendements using pull requests. Problems could be addressed through a mechanism of issues and fixes (patches). For every change it would be fully transparent who had proposed it (tracking). Legislative texts and snippets could easily be reused. And notarial deeds such as wills could be digitally signed and stored in a protected section of the GitLaw system. [1, 2]
It turns out that tech enthusiasts welcome the publication of any body of law or structured information on the legislative process, and are eager to explore how Git can be used to exploit this data. [1]
Top-down?
The bottom-up approach of a Git-like or Git-based system could provide a solid foundation to build more advanced functionality on.
The Greek 3GM project, however, shows that a top-down approach can be used too. This GSoC 2018 project parsed, analyzed and compared laws and amendments from the Greek Government Gazette using Natural Language Processing techniques. That allowed them to have amendments automatically merged into the law in the correct order, providing a fully codified, current version of each law at any given moment. The tool also clustered the laws according to their content, and ranked them based on incoming references. [1, 2]
AT4AM (Automatic Tool for AMendments) does something similar for the European Parliament: it was developed to help creating, editing and managing amendments ("diffs"). [1, 2, 3, 4, 5] Just like LEOS, it is based on the Akoma Ntoso XML schema, part of the OASIS LegalXML initiative [1]. AT4LEX (Authoring Tool for Legal Texts) was later developed for the creation of initial report drafts. Both tools are part of the e-Parliament Programme, aiming to establish a fully digital legislative text production chain. [1, 2]
Clear and clean
From the above, it appears that Git after 20 years has not only fully matured but has also become the main versioning tool in modern software development and the foundation of a global collaborative ecosystem. In addition to Git's "distributed" nature, the initial code and concepts being clear and clean have undoubtedly contributed to its tremendous success. I remember that I was very impressed by the simplicity of the design and the clarity of the code,
Junio recalled in an interview on Git's 15th anniversary [1].
Yet the main reason for Junio to start contributing to Git – rather than any of the other open-source version control systems available at the time – was that he wanted Linus to return to his work on the Linux kernel as soon as possible. He recalls that quite a few developers from the kernel community were implementing Git features in a rather chaotic competition. Working harder and faster on well designed and well implemented features, and presenting these better than others, is what Junio believes was decisive in Linus picking Git's new lead developer.
The best is yet to come
Despite everything that has already been accomplished, Junio said he believes the best features are yet to come – and we tend to agree. Git's impact on AI alone shows how relevant this versioning system is even at the forefront of today's most advanced technological developments. A notable example is Hugging Face, which is currently building a new ecosystem for machine learning, rooted in Git and FOSS principles. Above you could read about the idea of "GitLaw" and how Git's versioning concepts could be applied to legislative texts. Here below you can read how Git could be extended in various ways using domain-specific languages. Still, we are convinced that with the ideas presented in this article we have only just began to scratch the surface: the best is yet to come!
Extending Git using Domain-Specific Languages
A highly valuable extension would be to make the Git versioning system suitable for all types of files, including non-textual ones that currently can only be handled (stored) as monoliths. Making them "diff-able" in a meaningful way would unlock these files to the full capabilities of Git.
Bringing non-textual files under the Git versioning regime could best be achieved by inserting little codec layers into the Git system. Using Domain-Specific Languages (DSLs) for this would allow Git's functional domain to be extended in various important ways:
- Introducing a textual DSL to represent application-specific, non-textual data would allow for easy translation back and forth between the two formats. Instead of storing the opaque, non-textual version, Git would store the declarative DSL equivalent and be able to manage the data based on meaningful, line-by-line differences. In this setup, the original non-textual format would only be used for import/export-like functionality.
Using a textual DSL for data storage to enable high-granularity versioning is a proven approach. [1] - A DSL could also be used to provide an interface that protects sensitive data within a private repository by filtering out confidential information before it can be accessed by the user.
In this case, a DSL would be used to specify precisely what policies to be applied. Nescio is an example of such a filtering DSL (for network traffic). [1] - A similar approach could be used to make privacy-sensitive data from a private repository available to others. For example, access to anonymized patient records could be granted to researchers without violating GDPR/privacy rules.
Again, the privacy policies specified in the DSL would be part of the private repository and enable responsible access.
In all of these cases, DSL code would specify desired policies that can be attached to Git hooks [1] on the server.
Nederlands (nl-NL) 
English (United Kingdom) 
Add comment