No challenge solutions or flags here, and the underlying theory only where it’s unavoidable. This is about what you need when you understand the concept and the code still does nothing.

When you understand it and it still doesn’t work

I wrote something like twenty programs going through the Microarchitecture Exploitation dojo. Most of them did nothing the first time.

Not because the concept was unclear. A data load completes before the permission check does, and instructions speculatively executed in that gap leave traces in the cache. Train the branch predictor and code past a bounds check runs for a moment. Read the explanation and it makes sense. Write the code with that understanding and nothing comes out.

That’s where the real problem starts. With ordinary exploitation you can see why something failed. Attach a debugger, set a breakpoint, dump memory. None of that works here. Registers during speculation aren’t architectural state, so you can’t observe them. Cache state can only be inferred indirectly from access timing, and that inference is buried in noise.

So writing code here is less like building a program and more like building an instrument. Before you write code that produces a result, you write code that tells you why it failed.

This is what I picked up while building that instrument.

Only the gate changes

Work through enough of these and one skeleton shows up.

What causes the access that shouldn’t happen (the gate), how the value obtained that way turns into an observable side effect (the gadget), how you read that side effect (the channel), and how you recover it reliably out of the noise (the signal processing).

Of those four, only the gate changes from problem to problem. Spectre v1 is a conditional branch, v2 is an indirect branch target, Meltdown is a permission-violating load. The other three get reused as is. A Flush+Reload harness you build once keeps working while you swap gates underneath it.

How literally this holds becomes obvious when you put the v1 code and the v2 code side by side. Across two challenges against a kernel module, the difference on my side was one line.

1

buf[1] = 50;   // BTB training count

Channel, scoring, calibration, retry logic — none of it changed. Whether you’re fooling a conditional branch or an indirect branch target gets decided inside the kernel module, and all I did was pass along a parameter.

Once you see this, the question you ask of a new problem collapses into one. Where’s the gate here?

And the gate usually leaves hints in the code. If something goes out of its way to clflush the operand a bounds check depends on, that’s telling you to slow down the check and widen the speculation window. If a function pointer gets flushed right before an indirect call, that’s pointing at BTI. If a kernel module touches a secret but offers no legitimate path to read that value back, that asymmetry itself is the design saying to use a side channel.

Sometimes the gate isn’t native code at all. In challenges built around a custom VM, the gate lives inside that VM’s syscall implementation, and my job becomes writing a program in its instruction set. Open a file, read it, and call the instruction that uses a secret byte as an index.

1
2
3
4
5

prog += sysop(SYS_OPEN, 'a')
prog += sysop(SYS_READ, 'd')
patch_off = len(prog) + 2      # only this immediate byte changes each round
prog += imm('a', buf_off)
prog += sysop(SYS_EXEC, None)

Leaking a byte doesn’t mean rebuilding the program, just patching one immediate. When the gate is expressed as data, the attack loop gets that much lighter.

Developing an eye for these hints shortens the reversing phase considerably. Miss them and you can recover an entire VM ISA and still be stuck with no idea what to attack.

When you have to build the gate yourself

Meltdown needs no gate construction. Read a kernel address and you get a fault, and that fault is the gate. Spectre is different. You have to set up the situation so the predictor guesses wrong.

My first attempt had no training at all. It just wrote the target index and woke the victim, over and over.

1
2
3
4

*index = i;
for (int j = 0; j < 256; j++) _mm_clflush(probe + j * 0x1000);
sem_post(sem);
usleep(1000);

Nothing came out, which makes sense. From the predictor’s point of view every round looks identical, and there’s no reason for it to guess wrong.

The next version added training. Lean the predictor with a fixed index several times, then slip the target index in on the last iteration only.

1
2
3
4
5
6
7
8

int training_loops = 10 + (rand() % 10);
for (int t = training_loops; t >= 0; t--) {
    _mm_clflush(index_ptr);
    if (t == 0) { /* flush the whole probe array only now */ }
    *index_ptr = (t == 0) ? i : TRAIN_IDX;
    sem_post(sem);
    ...
}

Three things were intended here.

Randomizing the training count avoids the predictor learning the fixed-length pattern itself. Repeat exactly ten times and it has room to learn that the eleventh is different. I never measured this change in isolation, so I can’t say how much it contributed.

Flushing the probe array only at attack time also has a reason. Flushing all 256 pages on every training round disturbs the cache on its own and eats time. Training only needs to touch the predictor, so leaving the cache alone is better.

The one with the clearest effect was _mm_clflush(index_ptr). The victim’s bounds check depends on that value, and if it isn’t cached the check takes a while to resolve. That delay is the speculation window. This is the standard way to widen it for Spectre. Flip it around and it means: if challenge code goes out of its way to flush a check operand, the author left the window open for you.

There’s also a delay loop in this code.

1

for (volatile int d = 0; d < (100 + rand() % 100); d++);

Later I wrote something that looks exactly like this for Meltdown and it failed completely. The reason the same code split two ways is that the purpose differs. Here it’s wall-clock waiting for another process to speculate. There it was an attempt to stretch the speculation window inside my own instruction stream. The first works, the second doesn’t. More on that at the end.

If you can’t isolate failure, you can’t fix anything

My early code left me nothing to do when it failed. Nothing came out, and I couldn’t tell whether the channel was broken, the window never opened, or the address was wrong. So I changed something, it failed, changed something else, it failed.

Later I started splitting the code into modes.

1
2
3
4

#define MODE 2
// MODE 0 — verify the channel alone, pure userspace
// MODE 1 — verify fault suppression + speculative read (recover a known value)
// MODE 2 — actual kernel address read

MODE 0 failing means a cache channel problem. MODE 1 failing means fault suppression or the window. Only MODE 2 failing means the address or L1 residency. The failure point identifies the cause. Debugging time dropped by an order of magnitude once this was in place.

The same idea applies to picking a target. Don’t go for the flag on your first kernel read. If nothing comes out you won’t know whether that’s failure or just how it is. Read something whose answer you already know instead. The comm field of task_struct holds the process name in ASCII, so the moment babyarch_molten appears on screen, the gate and the address and the channel are all verified at once.

This works even where permissions prevent attaching a debugger. I know my own process’s name.

Once one byte reads, the rest is ordinary work

After comm shows up, exactly one thing is in hand. A function that reads one byte from a kernel address.

1

uint8_t leak_byte(uint64_t kaddr);

Call it eight times and you have a qword, and at that moment kernel memory becomes an array you can read. Pulling mm out of task_struct and pgd out of mm is just offset arithmetic. Ordinary pointer chasing.

This is where I think the structural character of these challenges lies. The technique writeups end at “you can read kernel memory,” but the actual target is usually in another process’s user memory. What bridges the two isn’t the side channel, it’s knowledge of kernel data structures. Meltdown is only the first link, and the rest is work anyone who knows Linux memory management would do.

The confusing part is that three kinds of addresses get mixed together. There’s the user virtual address the victim sees, the kernel virtual address you can hand to leak_byte, and the physical address stored inside page table entries. Fail to keep them separate and you lose track of which one a given line is dealing with.

The direct map bridges them. Linux linearly maps all of physical memory into the kernel address space at boot, so PAGE_OFFSET + physical is a readable address. The moment “knowing the physical address means being able to read it” holds, the walk becomes possible.

The walk itself is boring. Slice nine bits out of the virtual address for an index, pull the next table’s physical address out of the entry, repeat four times. It’s what the CPU does in hardware, transcribed into C.

The hard part is elsewhere. Every read is probabilistic. One wrong byte at one stage and the next table’s address is wrong, and everything below that is garbage. With no error message. Just odd values.

So verification goes in at every stage. Is the present bit set, does the physical address fall inside this system’s memory, does reading the same address repeatedly give a consistent value. You need at least this much to tell a missing mapping from a failed read when an entry comes back zero.

Don’t guess struct offsets. They differ by kernel version, one wrong offset collapses everything below it, and tracking down why is miserable. If the module kept debug info, pahole settles it, and otherwise /sys/kernel/btf/vmlinux does. It’s a five-minute job with no reason to skip it.

Put another way, this whole chain rests on a single primitive. Which makes it right to spend time hardening that one thing. Everything below is about that.

What to look at when the signal is weak

Always printing the histogram turned out to help a lot. The final verdict alone tells you nothing, but seeing which page got how many votes per round shows you the shape of the failure.

No votes at all means the gate never opened. Votes scattered around means the signal is there but weak, and that you can beat with repetition. Votes all piled on page 0 is the classic look of a register zeroed after a fault. The value didn’t leak, it read as zero, so dropping page 0 from the count surfaces the real signal buried underneath.

Changing the metric mattered too. At first only the fastest page per round got a vote.

1
2
3

if (t < min_time && j >= 0x20 && j <= 0x7e) { min_time = t; best_j = j; }
...
if (best_j != -1) hit_counts[best_j]++;

When the signal is weak, noise keeps stealing that single vote. Switching to accumulating every page under the threshold made a visible difference.

1

if (t_val < threshold) score[j]++;

The secret scores every time the gate succeeds while noise is sporadic, so the gap widens as trials pile up. I also raised attempts from 50 to 500 and then 1000, but that increase only started meaning something after the switch to accumulation.

Knowing the domain cuts noise at the source

If you know something about what you’re reading, using it as a filter makes whole categories of noise disappear.

A flag is printable ASCII. So candidate pages can be restricted to 32 through 126, and upper-byte noise never gets a chance to steal the argmax. Fewer lines to probe and flush is a bonus that makes each round faster.

Pointers and page table entries have stronger structure. Page-aligned means the low twelve bits are zero, a direct-map address has a fixed upper byte, and the present bit has to be set. Knowing these invariants lets you catch single-byte errors automatically.

There was a moment this actually worked as a detector.

1
2
3

[!] mm/pgd bad (mm=0xffff88807d4dd001 pgd=0xffff88807c66c0) → recalibrating
[calib] hit~72 miss~284 threshold=156
[*] mm=0xffff88807d4dd000 pgd=0xffff88807c66c000

mm has a stray low bit set and pgd is missing a byte entirely. Both tripped the “a kernel pointer can’t look like this” condition, recalibration fired, and the next attempt came back clean.

The channel and the exit are separate problems

Use Flush+Reload long enough and it starts to feel like the only option. But how you read cache line state and how you get the recovered value out are two different problems.

In one challenge I used the latency of prefetcht0 itself as the channel. Mapped and unmapped pages take different amounts of time to prefetch, and that’s enough to sweep the address space.

1
2
3
4
5
6
7

    rdtsc
    mov r12, rax
    prefetcht0 [rbx]
    rdtsc
    sub rax, r12
    cmp rax, r14
    jl  verify_address

What this yields isn’t data, it’s an address. The same tool doing both data leaks and address discovery makes sense once you think about it. Anything that leaves a trace in cache or TLB state can be a channel, and the only difference is whether that trace tells you a value or tells you something exists.

When a candidate turns up it isn’t accepted immediately. It gets confirmed five more times.

1
2
3
4
5

verify_loop:
    ...
    cmp rax, r14
    jg  next_page
    loop verify_loop

Sweep an entire address space and something will come back fast from noise at least once. Coming back fast five times in a row is far less likely. In scan-like work with many candidates, without a confirmation step the false positives dominate the result.

The exit was more extreme. With no shared memory available, bytes came out one at a time through the process exit code.

1
2
3
4
5
6

found_flag:
    mov rsi, {flag_offset}
    movzx rdi, byte ptr [rbx + rsi]
exit_syscall:
    mov rax, 60
    syscall

One new process per byte. Terrible bandwidth, but better than having no channel at all. And once you have this perspective other things start looking like exits too. Exit codes, running time, signal types, a line in a log.

Choosing the threshold worked differently here as well. Instead of deriving it by measurement, a candidate list gets tried in full and whichever one produces plausible output wins.

1
2
3
4
5

threshold_candidates = [85, 90, 95, 100, ..., 150, 75, 80]
for trial_threshold in threshold_candidates:
    char_code = extract_flag_char(i, trial_threshold)
    if char_code is not None and 32 <= char_code <= 126:
        break

The fact that a flag is printable ASCII acts as the threshold selector. A usable workaround in an environment where calibration isn’t convenient. It only holds when the domain constraint is tight enough, though, since a loose check lets a wrong threshold pass by chance.

Calibration isn’t a constant

Early code had this line in it.

1

uint64_t min_time = 150; // Threshold: adjust between 100~200 depending on the system

The comment already states the whole problem. It knows the value is system-dependent and hardcodes it anyway. Later this became a measurement at the start of each run, with a guard that falls back to a default if the result is absurd.

1
2

threshold = calibrate(buf);
if (threshold < 50 || threshold > 500) threshold = 150;

None of that was hard. What I didn’t know was that it drifts within a single run.

Code that ran fine with a threshold of 174 suddenly measured 287, and everything read after that was garbage. Adding a warmup and taking a conservative value across repeated measurements reduced how often this happened, but never eliminated it.

Using the median instead of the mean helped too. Timing distributions have long tails, and one interrupt drags the mean up wholesale.

1
2

hit_med, miss_med = statistics.median(hit_samples), statistics.median(miss_samples)
threshold = (hit_med + miss_med) / 2

There’s also the option of changing what you calibrate against. Rather than measuring HIT with a synthetic access, run the actual attack path with a guessable answer. If the flag starts with pwn.college{ then the first character is p, and how long that page takes to land in cache is the HIT for this specific path. A wrong guess means wrong calibration, but when synthetic measurement and the real path have different characteristics this is the more accurate one.

So I wired the structural check above in as a recalibration trigger. If the result is structurally impossible, measure again and retry. Watching recalibration show up in the log and the next read come back clean was fairly satisfying.

And then that same mechanism came back to bite.

When automatic recovery becomes an infinite loop

I once pinned the victim process to the same core as the attacker, thinking shared cache might help.

The opposite happened. Reads that had been working fell apart. The victim’s busy loop kept destroying the attacker’s cache state, so nothing came out starting from the mm and pgd stage. Removing the pin fixed it immediately.

This is where the recalibration logic spun forever. The structural check kept failing so it kept recalibrating, but the cause was cache interference rather than calibration, so no amount of remeasuring was ever going to fix it.

Recalibration wasn’t the wrong idea. But an automatic recovery mechanism is only valid for the failure it was designed for. Meet a failure with a different cause and it quietly becomes an infinite loop, which makes finding the real cause take longer than it would have. Automation that hides failure is worse than no automation.

Code I wrote for a different challenge happened not to have this problem. It caps retries, and past the cap it accepts the best it has along with a marker.

1
2
3
4
5

if (retry >= 5) {
    flag[i] = (char)best;
    printf("Index %2d: %c  (WEAK)\n", i, flag[i]);
    break;
}

The (WEAK) marker is the point. Take the value but record that confidence was low. Then you can look at the whole result and see which positions are suspect, and if the surrounding characters give you context you can fill it in by hand. A compromise that neither stalls nor hides the failure.

The more important instinct this left behind is to always be aware of who fills the cache and who destroys it. Co-location helps in some situations and hurts in others.

To be straight about one thing: attacker-side core pinning was on from the first run and I never compared with it off. The case for pinning is theoretical, and I didn’t measure the difference in this environment.

The problem nobody explains in detail

Meltdown only reads what’s in L1. If the target sits in L2 or memory, the permission check resolves before the data arrives.

Explanations note this condition in one line and move on. In practice it’s where you get stuck most. That byte you want, right before you read it, how do you get it into L1?

The answer differed every time. If some kernel or VM logic walks over the secret, call it and let it do the reading for you. If there’s an ioctl that dereferences a user pointer, the cache line stays in L1 even though no value comes back, which makes it a preloader in effect. Before and after adding that preload, the numbers went to 1991 matches out of 2000.

Some interfaces make the kernel walk page tables, which indirectly pulls the line you want into cache. Except when the permission check fires before the walk does, targeting a setuid process for instance, in which case it does nothing at all. Fail to distinguish this failure from the others and you’ll be lost for a while.

If the victim process is still running you can lean on that directly, since it keeps its own data in its own core’s cache. x86 L1D is physically indexed, so the same physical line is the same entry regardless of which virtual address cached it. That’s why what the victim brought in through its own user VA is readable by me through a direct-map address.

How to prime is ultimately a question of who in this system touches that memory. And you can’t answer that without understanding the system’s structure. This is the point where a side channel is a systems problem rather than a pure hardware one.

C source is not a contract with the CPU

The thing I fought longest and learned the most from.

To widen the speculation window I wrote this.

1
2
3
4

volatile double d = 1.0;
for (int i = 0; i < 40; i++) d /= 1.7;
uint8_t v = *(volatile uint8_t *)kaddr;
(void)probe[v * 4096];

Floating-point division is slow, so the permission check would be delayed and the window would widen. That was the reasoning. The histogram came back completely empty. Changing the iteration count didn’t help. Adding more volatile didn’t help.

Switching to inline assembly made it work immediately.

1
2
3
4
5
6

    xorq %rax, %rax
1:
    movzbq (%[addr]), %rax
    shlq $12, %rax
    jz 1b
    movq (%[buf], %rax), %rbx

Same intent, different outcome. A compiler is free to reorder instructions, delete them, and reassign registers as long as meaning is preserved. What volatile guarantees stops at “don’t remove this access.” It says nothing about how surrounding instructions get placed.

Which is obvious in hindsight. C’s semantics are defined over an abstract machine, and that abstract machine has no speculative execution, no cache, no branch predictor. Everything I was trying to manipulate lives below that abstraction. I was trying to steer the bottom from the top.

There are points in this area where inline assembly isn’t a preference but a requirement. And more important than that is the habit of checking the critical loop with objdump -d every time. Comparing what I wrote in C against what the CPU will actually execute. Without that habit you never find out why it doesn’t work.

Worth noting why the jz 1b retry loop is effective, too. Putting a branch that loops again when the value is zero gives the CPU more chances to reach the probe access while speculating through it. It isn’t widening the window so much as getting more attempts inside it.

What’s left

I don’t think what I got out of this dojo was side-channel knowledge. That’s available everywhere if you go looking.

What’s left is a way of handling things you can’t observe. Verify against something whose answer you know. Structure the code so failure identifies its cause. Derive constants by measuring instead of guessing. Represent uncertain results as “unknown.” When something doesn’t work, remove rather than add, and change one thing at a time.

None of this is specific to side channels. It transfers directly to any work involving uncertain observation.

One more thing. The perspective of pushing a system into a state its developers never considered keeps showing up outside this area. Kernel races do it along the time axis, and smart contract invariant violations do it along the call-ordering axis. Only the axis differs, and the question being asked is the same. Where are the states this machine can reach that nobody thought about?