Your Laptop Memory Is A Lie

When you first study to program, reminiscence appears fairly simple. You have a bunch of bytes, every with an tackle. If you need to learn or write one thing, you specify the tackle, and the pc goes to that place and does your bidding. Sure, some addresses could be reserved for {hardware} or the working system, however the common thought is straightforward: reminiscence is a giant array of bytes.

Just like we fake {that a} wire is an ideal conductor, the large array of reminiscence mannequin was a slight twisting of the reality. On a contemporary pc, nonetheless, it’s virtually fully a lie. The tackle you employ most likely isn’t a bodily tackle. The knowledge you learn may not come from RAM in any respect. Even if it does, it could be sitting in one among a number of caches. And should you observe the rabbit gap far sufficient, you uncover that your SSD is arguably simply one other degree within the reminiscence hierarchy.

The Need For Speed

The basic downside is that processors have gotten a lot quicker than reminiscence. A contemporary processor can execute a number of directions per clock cycle, however accessing strange DRAM can take a whole lot of cycles. If the processor needed to await RAM each time it wanted knowledge, a lot of that spectacular computing energy would go to waste.

The resolution is caching. Keep copies of regularly used knowledge in smaller, quicker reminiscences nearer to the processor. Most fashionable desktop processors have at the very least three ranges of cache. The L1 cache is tiny however extraordinarily quick, and sometimes cut up between directions and knowledge. L2 is bigger and considerably slower, whereas L3 is even bigger and often slower as properly.

Things get fascinating with a number of cores, as is usually the case lately. Typically, every core has its personal L1 and L2 caches, whereas a number of cores share an L3 cache. But that’s not common. Some processors share L2 caches between cores. Others have a number of L3 caches, every serving a gaggle of cores. The actual association relies on the processor structure.

One vital element: caches don’t typically work byte by byte. They switch and monitor blocks referred to as cache strains, generally 64 bytes on fashionable desktop processors. If you learn one byte that isn’t cached, the processor typically fetches your complete line containing that byte.

This is why sequential reminiscence entry is usually a lot quicker than random entry. If you learn the following byte, there’s a great probability it’s already within the cache. Hardware prefetchers might even anticipate what you’ll want subsequent.

But there might be issues. Imagine two cores modifying utterly totally different variables. If these variables occur to occupy the identical cache line, the cores might repeatedly invalidate one another’s copies, losing time regardless that they aren’t logically sharing knowledge.

Protocols comparable to MESI hold the caches coherent, coordinating which core owns a line and which copies are legitimate. MESI is an acronym for cache line states: modified, unique, shared, or invalid. Memory techniques should work out methods to hold caches in sync. But coherence doesn’t eradicate the necessity for synchronization or memory-ordering guidelines in multithreaded applications. The level is that even our easy byte-addressed reminiscence is secretly working in bigger chunks.

The Address Isn’t Real, Either

So far, we’ve assumed that the tackle you provide identifies a location in RAM. Usually, it doesn’t. Modern working techniques use digital reminiscence. Your program sees a digital tackle area that the working system and processor cooperate to map onto bodily reminiscence.

The reminiscence administration unit, or MMU, interprets digital addresses into bodily addresses, usually utilizing web page tables. This lets totally different applications use equivalent digital addresses with out interfering with each other. It additionally supplies reminiscence safety and permits bodily pages to be shared.
Of course, translating addresses takes time. So we add one other cache.

The translation lookaside buffer, or TLB, remembers not too long ago used tackle translations. A TLB hit avoids a probably costly page-table stroll. Some processors even have a number of ranges of TLBs. Unlike the strange CPU cache, which typically operates on cache strains, digital reminiscence often works with pages, usually 4 KB, though bigger pages are additionally supported.

Notice what’s taking place. We have caches for knowledge, caches for directions, and caches for the data wanted to find the information and directions.
The easy act of studying a variable has change into surprisingly difficult.

Not All RAM Is Created Equal

Even if we get by means of the caches and at last attain RAM, issues aren’t essentially easy. On techniques with a number of processors or sure multicore preparations, reminiscence entry might be non-uniform. This known as NUMA, or Non-Uniform Memory Access.

A processor might have comparatively quick entry to reminiscence connected to its personal NUMA node, whereas accessing reminiscence related to one other node requires crossing an interconnect. Both places are strange RAM, and each might seem in the identical tackle area, however the entry instances and out there bandwidth can differ.

The working system tries to put reminiscence close to the processors utilizing it. Moving a thread to a different CPU can subsequently change its reminiscence efficiency, regardless that this system’s addresses haven’t modified.

And NUMA isn’t restricted to conventional multiprocessor servers. Modern techniques with chiplets, heterogeneous reminiscence, and applied sciences comparable to CXL are making reminiscence topology more and more fascinating.

What If It Isn’t In Memory?

Virtual reminiscence has one other trick. A digital web page doesn’t essentially must be backed by bodily RAM in the intervening time you entry it.

When a program accesses a web page that isn’t presently mapped in a usable approach, the processor generates a web page fault. The working system decides what to do. Sometimes it merely allocates a recent web page. Sometimes it retrieves knowledge from an executable or memory-mapped file. Other instances, the web page was moved out of RAM to swap area and have to be retrieved. Note that issues like executable knowledge are typically immutable, so most working techniques gained’t swap them out. They merely trash them and know they’ll reload the identical web page later when wanted.

This means your program can tackle knowledge that isn’t presently in bodily reminiscence. Naturally, accessing an SSD is far slower than accessing RAM, so extreme swapping can convey a pc to its knees.

Virtual reminiscence doesn’t really require swapping, although. Address translation and safety are helpful even on techniques that by no means transfer pages to disk.

Your SSD Is Memory, Too

Suppose your program reads a file. You may think that the working system sends a request to the SSD and waits for the information. But if the file was not too long ago accessed, the information might already be in RAM.

Operating techniques preserve a filesystem web page cache, retaining regularly or not too long ago accessed file knowledge in reminiscence. Writes can also be buffered in RAM earlier than ultimately reaching persistent storage.

This makes RAM act like a cache for secondary storage, a lot as L3 acts like a cache for RAM.

The analogy isn’t excellent. RAM can be the first working storage for operating applications, and nameless reminiscence isn’t essentially a replica of something on disk. But the final hierarchy is evident: registers, CPU caches, RAM, and chronic storage present progressively totally different tradeoffs between pace, capability, and value.

Even SSDs have their very own caches and tackle translations. Their flash translation layers map logical block addresses onto bodily flash places, hiding put on leveling and rubbish assortment from the host. Apparently, no one needs to inform the reality about addresses.

Compressing The Middle

But what if we may make RAM successfully bigger with out including extra? Compression provides one other technique to complicate our reminiscence hierarchy. Instead of instantly transferring an sometimes used web page to disk, why not compress it and hold it in RAM?

Linux has two well-known mechanisms for doing this: zram and zswap. Zram creates a compressed block system in RAM. It’s generally used as a swap system, permitting pages to be saved in compressed kind moderately than written to bodily storage.

Zswap, alternatively, is a compressed cache in entrance of a swap system. When Linux decides to swap out a web page, zswap makes an attempt to compress and retain it in RAM. If essential, pages can ultimately be evicted to the backing swap system.

Both commerce processor time for reminiscence capability and decreased storage I/O. Decompressing a web page prices CPU cycles, however that may be significantly cheaper than retrieving it from an SSD.

There’s a catch, although. These mechanisms function by means of the swap subsystem. When a compressed web page is required once more, it have to be decompressed into an strange web page earlier than this system can entry it. That means web page faults, reminiscence administration overhead, and transferring knowledge round. But what if compressed reminiscence may behave extra like strange RAM?

Meta Wants To Change The Rules

That’s the concept behind CRAM, or Compressed RAM, a venture introduced by Meta’s Gregory Price on the October 2026 Linux Plumbers Conference.

Rather than treating compressed reminiscence as a traditional swap system, CRAM proposes exposing it by means of a particular NUMA node. Linux can then use present memory-management mechanisms to maneuver pages between strange DRAM and compressed reminiscence. The most fascinating half is the hardware-assisted compression and decompression, which permits entry at a lot finer granularity than standard page-oriented compressed swap.

The builders report spectacular benchmark outcomes, together with learn efficiency that approaches strange DRAM in some checks and dramatic enhancements over zram for chosen workloads.

That doesn’t imply compression has abruptly change into free. Writes, compression ratios, {hardware} necessities, and managing incompressible knowledge all matter. CRAM continues to be developmental work, not prepared for on a regular basis use.

But it illustrates the place the reminiscence hierarchy is heading. Instead of merely deciding whether or not a web page is in RAM or on disk, the working system might select amongst strange RAM, compressed reminiscence, slower reminiscence tiers, and secondary storage.

The Illusion Is The Feature

Most programmers don’t want to consider any of this more often than not. That’s the purpose of an abstraction. But when efficiency issues, the phantasm begins to leak. Sequential entry can beat random entry by monumental margins. Two threads modifying unrelated variables can combat over cache strains. Moving a course of between CPUs can change its reminiscence latency. A web page that appears like strange reminiscence would possibly really require decompression or a visit to an SSD.

On smaller embedded techniques, in fact, the image could also be very totally different. Without an MMU or elaborate caches, reminiscence can look way more like the easy bodily tackle area we discovered about within the first place.

We began with an array of bytes. We ended up with a hierarchy of caches, translation tables, NUMA nodes, compressed reminiscence swimming pools, and chronic storage units, all cooperating to take care of that easy phantasm. Amazing how difficult one thing so easy might be.