When data arrives over a network, does the CPU keep reading bytes directly from the cable? A typical OS divides the work into commanding the device, transferring data into memory, and reporting completion. Devices continue operating while the CPU computes, so the system must decide who may access each memory region and when.
Issuing commands and reading device state
Network cards and storage devices have controllers that operate the hardware. A driver starts operations by writing values to control registers and reads registers to check status. Memory-mapped I/O maps part of the address space to these registers.
Although CPU instructions may look like ordinary loads and stores, device registers do not behave like normal RAM. Reading a register may acknowledge a notification; changing the order of writes may change the device's behavior. Real drivers follow the device specification and use the OS's I/O-access APIs. Ordinary pointer operations are not always a valid substitute.
Choosing how to detect completion
Polling repeatedly reads status to check whether an operation has finished. It consumes CPU time while waiting, but it can be useful when short waits must be detected with low latency. With interrupts, a notification from a device or another source causes the CPU to transfer control to a handler. Constant monitoring is unnecessary, but entering and leaving a handler also costs time.
Distinguish interrupts from external devices or timers from exceptions related to an executing instruction. An access to a page that is not present, for example, causes an instruction-related page fault. Both can transfer control to the OS, but their causes and the rules for returning to the interrupted instruction need not be the same.
An interrupt handler identifies the event, acknowledges the device, and arranges any follow-up work. Raising an interrupt for every packet can be expensive at high traffic rates. Systems may batch notifications or use an interrupt to begin processing a batch of packets by polling. Interrupts and polling can be combined.
DMA transfers data without a CPU copy for every item
Direct memory access (DMA) lets a device or DMA engine transfer data to or from memory instead of making the CPU copy each item with loads and stores. The CPU prepares buffers and transfer information, then handles the result after completion. DMA does not eliminate transfer setup or completion processing.
Allocate a receive buffer
→ Set up a DMA address usable by the device
→ Give the buffer to the device
→ The device writes data
→ Notify/check completion
→ The CPU processes the buffer contentsA CPU virtual address cannot necessarily be passed directly to a device. DMA addressing depends on matters such as the device's address width and the IOMMU. The Linux DMA API handles mappings and any necessary cache synchronization. Buffer ownership also matters: the device must not overwrite a buffer while the CPU is reading it.
Keeping multiple cores' caches coherent
If each core has its own cache, the system needs a mechanism to stop one core from indefinitely reading a stale copy after another core writes a location. Cache coherence manages copies of the same memory location. It does not, by itself, resolve memory-ordering questions about the order in which other cores observe operations on different locations.
When two threads increment a shared counter, coherent cache values do not make the read, addition, and write a single operation. If both threads read 5 and both write 6, one increment is lost. An atomic read-modify-write instruction or an appropriate lock is needed to protect the full operation.
Even different variables can interfere when they occupy the same cache line. Writes by separate cores can make the line move between caches and reduce performance. This is false sharing. Distinguish correctness from performance, measure the effect, and then reconsider data layout or how work is divided.
Check your understanding
Question 1: Does DMA mean the CPU has nothing left to do? No. It still prepares buffers, configures the transfer, and handles completion, among other tasks.
Question 2: Does cache coherence make an ordinary increment safe for a shared counter? No. Coherence does not guarantee that the whole sequence from reading to writing is indivisible.
To trace how operations interleave and how to protect them, continue to Interleaving, Race Conditions, and Atomicity.