Ridge-point workbook

A ridge point is two spec-sheet numbers divided. The work is not the division. The work is trusting it enough to throw away a clever idea.

Spec sheets (use these, not a wiki)

Card	Peak FP16 tensor	Peak bandwidth	Memory
T4	65 TFLOP/s	320 GB/s	16 GB
L4	242 TFLOP/s	300 GB/s	24 GB
A100 80GB SXM	312 TFLOP/s	2,039 GB/s	80 GB
H100 SXM	989 TFLOP/s	3,350 GB/s	80 GB

Treat 1 TB/s as 1000 GB/s here so the arithmetic stays in one unit. (Binary vs decimal is a later fight. Consistency is the current one.)

Part A — Four ridges

For each card compute:

ridge  =  peak_FLOP/s  /  peak_bytes/s

Report FLOP/byte, rounded to the nearest integer. Rank the cards from "easiest to become memory-bound" to "hardest."

Then answer, in two sentences: why is L4 more memory-bound-friendly than A100 even though A100 has more bandwidth?

Part B — Six kernels, one table

Compute arithmetic intensity, then mark each kernel below or above each card's ridge.

Use fp16 = 2 bytes. Traffic is a simple streaming model: every input read once, every output written once. No cache hits. That is the conservative (memory-heavier) estimate, which is the one you want when deciding whether clever math can help.

kernel	FLOPs	bytes moved
x + y, 4096×4096	n²	3 n² · 2
x * 2 + 1 fused, 4096×4096	2 n²	2 n² · 2
x @ y, n=512	2 n³	3 n² · 2
x @ y, n=4096	2 n³	3 n² · 2
decode 7B, batch 1	2 × 7e9	14e9
decode 7B, batch 32, ignore KV	32 × 2 × 7e9	14e9

Fill:

kernel	intensity	T4	L4	A100	H100

For the last row, explain in one sentence why ignoring KV is a lie that gets worse as batch and context grow.

Part C — Forbidden lists

For L4 and for H100, write a forbidden-optimisation list of three items each: techniques that would be a waste on that card for decode at batch 1. Then write a permitted list of three.

The lists should differ. If they do not, you have not used the ridge.

Part D — A wrong blog

A blog reports:

Our new GELU is 4.2× faster than PyTorch.

No shape. No dtype. No card. No fused-vs-eager. No bandwidth or TFLOP/s.

Write the three questions you would email the author. Then write the one-sentence rule this course uses instead of "X is faster than Y."

Rules

1. Show the arithmetic. A table of answers with no working is not a pass.
2. Peak spec-sheet numbers are ceilings, not promises. You are classifying intent, not predicting achieved %.
3. Batching decode without adding KV traffic is an upper bound on how free batching is, not a measurement.

Acceptance

1. Ridges land near: T4 203, L4 807, A100 153, H100 295 FLOP/byte.
2. L4 ranks as easiest to be memory-bound (highest ridge).
3. Elementwise add is below every ridge. 4096 matmul is above every ridge. 512 matmul is close enough that you must say "it depends" and why (launch + intensity both in play).
4. Batch-1 decode is below every ridge.
5. The blog critique refuses to accept a speedup without a shape.

Stretch

Look up one more card you actually use (4090, H200, B200, whatever is on the desk). Compute its ridge from a vendor PDF, not a tweet. Place decode of 70B int8 (70 GB of weights) on it.

Check your work

Ridges. T4: 65e12 / 320e9 = 203. L4: 242e12 / 300e9 = 807. A100: 312e12 / 2.039e12 = 153. H100: 989e12 / 3.35e12 = 295.

L4 has a thin bus relative to its tensor FLOPs, so the ridge sits high: almost everything ordinary is memory-bound. A100 has a fat bus, so you actually can get compute-bound on large GEMMs.

Add intensity. n² / (6 n²) = 0.167. Below all.

Fused affine. 2 n² / (4 n²) = 0.5. Below all.

Matmul 512. Intensity 2 n³ / (6 n²) = n/3 = 171. Below L4, near T4, above A100, below H100. Also small enough that launch overhead still shows. "It depends" is the honest cell.

Matmul 4096. Intensity 4096/3 ≈ 1365. Above all four.

Decode batch 1. Intensity 14e9 / 14e9 = 1. Below all.

Decode batch 32, no KV. Intensity 32. Still below L4 and H100 and T4; below A100 too. Batching helps utilisation of already-paid weight traffic, which is a bandwidth argument, not a ridge crossing — until KV is counted.

Debrief

The ridge is a gate. Below it, you are not allowed to talk about FLOPs as the thing you will improve. That sentence, used in a design review, is worth more than a kernel.