Memory-Mapped I/O vs Port-Mapped I/O

When the CPU wants to talk to a device, a network card, a disk controller, a keyboard, it does so by reading and writing the device’s registers: small storage locations inside the device for commands, status, and data.

The question is: how does the CPU address those registers? Historically there are two answers.

Port-Mapped I/O

In port-mapped I/O (also called isolated I/O), devices live in their own separate address space, made of numbered ports. The CPU uses special instructions to access them, on x86 that’s in and out:

1mov dx, 0x3F8   ; COM1 serial port
2mov al, 'A'
3out dx, al      ; write the byte to the device

This is the old-school way. The x86 port space is tiny (65,536 ports), and only privileged code can execute these instructions. Legacy devices like the PS/2 controller or the classic serial port still work this way.

Memory-Mapped I/O

In memory-mapped I/O (MMIO), the device’s registers are mapped into the normal physical address space. Some range of addresses simply doesn’t go to RAM, the memory controller routes it to the device instead.

The huge advantage: no special instructions. Talking to a device is just a load or a store:

1volatile uint32_t *status = (uint32_t *)0xFEB00000;
2uint32_t s = *status;   // reads a device register, not RAM

Any addressing mode works, any register width works, and drivers can be written in plain C. This is how essentially all modern devices work, your GPU, NVMe drive and network card all expose their registers through MMIO (this is what the BARs you see in lspci output are: the address ranges where a device’s registers got mapped).

The Catch

Because MMIO accesses look exactly like memory accesses, the hardware and the compiler will happily apply memory optimizations to them, caching, reordering, combining writes. For a device register, that’s a disaster: reading a status register twice should hit the device twice.

That’s why device memory has to be mapped as uncacheable, and why driver code is full of volatile and memory barriers. The pointer above without volatile would be a genuine bug, the compiler could optimize the second read away.

One thing that confused me for a while: MMIO is unrelated to mmap(). Same word, completely different levels, mmap() maps files into a process’s virtual memory, MMIO maps device registers into the physical address space. And for moving bulk data, the CPU doesn’t sit there doing MMIO loads in a loop, that’s what DMA is for. MMIO is for control, DMA is for data.