YonedaRepo Research

Source Code as a Transactional Abstraction.

Source code and source control are now transactional abstractions. Data lakes made the same move when Delta Lake and Spark turned files on object stores into tables with a transaction log, snapshots and optimistic concurrency; this series builds that foundation for source control.

  1. LogEvery ref change is one hash-chained entry in an append-only log.
  2. ValidateA landing is checked against every entry written since its base.
  3. DeriveIndexes, symbols and Git commits are views maintained from the log.

What we measured

The test: 1,000 agents writing to one branch of one repository at the same moment, on the reference build running on Cloudflare.

1,000 / 1,000
Every change landed

1,000 simulated agents pushed changes to the same branch within seconds. All 1,000 landed in each of 6 runs, and no request was refused. The previous design, which built Git trees while landing, got 4 of 1,000 through under the same load.

Short bursts. A sustained-load test is still to come.

0.7–1.7 s
Time to save a change

With 1,000 agents writing at once, 95% of commits were saved within this time.

Under 1 s in 2 of 3 runs. With one shared login for all agents it was 1.1–1.6 s.

3.2–5.4 s
When Git catches up

A change is final the moment it enters the transaction log. The Git copy that git clone sees is rebuilt in the background and trails by a few seconds.

On an earlier build the lag was 17 s.

Machine-checked

180 theorems
Checked by a computer, step by step

The core rules (how changes are logged, checked and merged) are written in the Lean proof language, and Lean verifies every proof. No step is skipped or taken on trust.

The proofs cover the design, not the running code.

Batch = one at a time
Landing many changes at once is safe

When the system lands a batch of changes together, including merged ones, the result is exactly what you would get by landing them one after another.

Holds when no change reads or writes a file an earlier change in the same batch wrote, and merges don't write back stale content. Merges are modelled line by line with stable line identities; shifting line numbers are not covered.

Bug found
Why the extra check is needed

Without a check between changes in the same batch, two changes can each pass on their own yet produce a result no one-at-a-time order could. The analysis in Part I found this, Lean confirms it, and the fix is now in the code.

Shown with a concrete example. The test workloads do not yet exercise the check.

Papers

Overview: start here

Source Code as a Transactional Abstraction

We argue that this is the next evolution in code, the move data lakes made from files to transactional tables, and assemble its foundations from a three-part series. The…

26 pagesRead PDF
Part I

A Transaction Log for Source Control

This paper gives the transaction model on which that view rests. In Git, when many always-online agents write against one moving branch, every lost compare-and-swap race on the…

35 pagesRead PDF
Part II

Partitioned Derived Data

This paper supplies the part of that foundation that concerns derived data. We model search indexes, symbol tables, blame and work plans as views over the transaction log…

36 pagesRead PDF
Part III

A Cloud-Portable Architecture for Serverless Git

This Part measures what that transaction system costs on cloud primitives and where its serialized authority saturates. The architecture needs an object store, one serialized…

35 pagesRead PDF
Part IV

Machine-Checked Batch Landing: Merge Resolution and Serializability in a Transactional Model of Source Control

Part I of this series gave the transaction model behind that view and proved its properties by hand. This paper makes the core of that model machine-checked. A Lean 4 development…

39 pagesRead PDF

How it works

Part I

The transaction model

A repository is a database, a change is a transaction with a read set and a write set, and a version is a snapshot. Commits are built on the server and recorded without moving any ref. A landing validates against every log entry written since its base and appends one hash-chained entry in the same atomic step as the ref update.

Part II

Derived data as views

Search indexes, symbol tables, blame and work plans are views over the log, maintained by partitioned jobs over immutable snapshots. A view is consistent at a snapshot whenever the key of each stored partition result determines that result.

Part III

Serverless realization

An object store, one serialized authority per repository, a durable queue and stateless compute are enough. The reference implementation runs on Cloudflare, and the paper measures where the authority saturates.

Part IV

Machine-checked batch landing

A Lean 4 development formalizes the core of the model and proves that an accepted batch, merges included, installs the serial execution of its members, with checked counterexamples showing that neither hypothesis can be dropped.

Overview

Git built after landing

Composed, the three parts let a branch's Git commit be a deterministic function of the log prefix, so Git is built after landing without weakening the log's guarantees.