Cache Coherence: How Cores Agree on Memory

A modern CPU has a private L1 and L2 cache per core. Which raises an awkward question nobody asks until concurrency bites them: if core 0 and core 1 both have a copy of the same memory in their private caches, and core 0 writes to it… what stops core 1 from reading a stale value forever?

The answer is the cache coherence protocol, a piece of hardware machinery that quietly underpins every mutex, every atomic, and every concurrency guarantee your language gives you. Back in December I wrote about false sharing and hand-waved “cache coherence” as the cause. This article is the follow-up: what the protocol actually does.

The Contract

Coherence gives you, roughly, one guarantee: for a single memory location, all cores agree on the order of writes, and a write eventually becomes visible to everyone. It makes the many private caches behave as if there were one shared memory.

Note what it does not promise: anything about the ordering of operations across different locations. That’s the separate (and messier) topic of memory consistency models, barriers, acquire/release and friends. Coherence is the foundation that memory models are built on, not a replacement for them.

MESI: Four States per Cache Line

Coherence is tracked per cache line (64 bytes), not per byte. In the classic MESI protocol, every line in every cache is in one of four states:

• M, Modified: this cache has the only copy, and it’s dirty (memory is stale). This core can read and write freely.
• E, Exclusive: this cache has the only copy, and it’s clean. Writes are free, the state silently upgrades to M.
• S, Shared: this line may also be in other caches. Reading is fine; writing is not allowed in this state.
• I, Invalid: the line is dead. Touching this data means fetching it again.

The whole protocol boils down to one invariant: either one writer (M/E), or many readers (S), never both. It’s a readers-writer lock, implemented in hardware, per cache line.

The Dance

The caches keep the invariant by exchanging messages (on a snooping bus in the simple design, via a directory on big multi-socket machines). The two interesting moves:

A core writes to a Shared line. It’s not allowed to, first it must broadcast an invalidate to everyone else. The other caches drop their copies (S → I) and only then does the writer proceed (S → M). This message is called an RFO, Request For Ownership, a name that shows up in every profiler, so it’s worth knowing.

A core reads an Invalid line that’s Modified elsewhere. The cache holding the dirty copy must supply the data (often via a direct cache-to-cache transfer), and both settle into Shared. The read still works, it’s just several times more expensive than a normal cache hit.

Line by line, state by state, this is all there is. Memory is coherent because every write forcibly evicts every other copy first.

This Explains Things You’ve Already Seen

Once you know about RFOs, a bunch of performance folklore becomes mechanical:

Why atomics cost. An atomic.AddInt64 isn’t expensive because of the addition, it’s expensive because the core must get the line into M state, and under contention, every core is constantly stealing ownership from the others. The counter’s cache line ping-pongs, and each pass costs a cross-core transfer (~tens of nanoseconds) instead of an L1 hit (~1 ns).

Why false sharing is a disaster. Coherence tracks lines, not variables. Two “independent” counters on one line share one M state, so two writers fight over ownership exactly as if they shared data. The fix is padding; the reason is MESI.

Why read-mostly scales beautifully. Any number of caches can hold a line in S simultaneously. A hundred cores can read the same configuration struct with zero coherence traffic. It’s writes to shared data, and only those, that serialize.

Why per-core sharding works. Structures like Go’s sync.Pool or per-CPU counters in the Linux kernel aren’t just avoiding lock contention, they’re keeping each line in M state on one core forever, which means no coherence messages at all. The fastest cache line is the one only you touch.

One Level Deeper: Store Buffers

There’s one more wrinkle worth knowing. Waiting for all those invalidations to complete on every write would stall the core, so cores don’t wait: writes go into a small store buffer first and retire to the cache later. The core keeps executing; its own reads check the store buffer so it sees its own writes.

The price: for a brief window, a write is visible to this core but to nobody else, which is precisely how you get the classic litmus test where two cores each write a flag then read the other’s, and both read zero. Not because coherence failed, but because both writes were still sitting in store buffers. Memory barriers exist almost entirely to flush this machinery on demand, that’s the memory-model rabbit hole, and it deserves its own article someday.

What I Took Away

• Coherence is a hardware readers-writer lock per 64-byte line: one writer or many readers.
• Every write to shared data is an ownership fight. Contention costs come from cross-core transfers, not from your arithmetic.
• The scalable patterns, immutable data, sharding, padding, all reduce to one rule: don’t make cores share writable cache lines.

The thing I find genuinely satisfying: locks, atomics, memory models, all of it bottoms out in a small state machine stamped next to every cache line, exchanging invalidate messages billions of times per second, in perfect silence.

Thanks for reading.