Scaling 30TB+ SMR: The Numbers Behind Our MSST 2026 Talk

Presented at MSST 2026, The International Conference on Massive Storage Systems and Technology, Santa Clara University, June 1, 2026

SAS4 performance gains, HAMR-SMR vs. ePMR-SMR, and the file-system lever nobody is pricing in

There is a sentence I have heard in every storage procurement conversation for the last five years: “SMR is too slow for serious workloads.” So we stopped arguing and started measuring. In our own lab, and in cooperation with PCSS (the Poznań Supercomputing and Networking Center), we built two reference platforms, populated them with 30 TB SMR drives from both major vendors, and ran an identical FIO sweep across every variable that matters. This June I presented the results at MSST 2026; this post is the write-up. The short version: the misconception is wrong. Paired with the right stack, 30 TB SMR nearly matches raw HDD throughput — and on writes, the right combination beats what most people think the drive can do.

“SMR is too slow for serious workloads.” We measured. It isn’t.

The work isolated three independent variables — vendor, interface, and HBA — plus one bonus that turned out to be the biggest single-day gain of the year: how the file system decides which zone to write next. Here is what each one moved.

Download the full deck: Scaling 30 TB+ SMR  – MSST 2026 slide deck — every chart, methodology slide, and raw curve behind the numbers below.

The test bench

Two reference platforms, two HDD vendors, two HBA vendors, EC6+2 and raw throughput throughout. Everything ran on LeilFS 5.10 (Stable and Experimental branches) over a dual-port 100 GbE QSFP28 fabric.

Pillar 1 – Vendor: HAMR (Seagate) vs. ePMR (WD)

At 30 TB the two recording technologies diverge, and they diverge most where it hurts: sustained writes. Running EC6+2 across eight drives on LeilFS 5.10 Stable, Seagate’s HAMR sustained roughly 1.45 GB/s of sequential writes against WD’s ~960 MB/s — a 50% gap. Reads were closer: Seagate held 1.77–1.82 GB/s versus WD’s 1.62–1.63 GB/s once the sweep saturated, about +9%.

The raw single-drive picture explains why. At 16 concurrent FIO jobs landing in different zones — the load pattern an EC chunkserver actually produces — Seagate held 192 MB/s per drive while WD collapsed to 112 MB/s, a 58% delta. Across the 1→16 job sweep, Seagate lost only 17% of its single-job bandwidth; WD lost 48%. The WD curve plateaus immediately because it is hitting the drive’s physical limit, not the file system’s.

Pillar 2 – Interface: SAS4 24G vs. SAS3 12G

24G isn’t just a faster cable — it changes the ceiling for high-density SMR. We ran 60 identical drives through the same FIO sweep on each fabric: SAS4 24G 2-port versus SAS3 12G 4-port. At 60 jobs, SAS4 delivered 15.3 GB/s reads against SAS3’s 13.3 GB/s (+14.7%) and 13.25 GB/s writes against 11.39 GB/s (+16.3%). Both interfaces scale linearly to about 40 jobs; past 20 jobs the SAS4 lead opens and holds. That is roughly 15% of free top-line headroom on the same drive sizing — reason enough to standardize on SAS4 for new builds.

Pillar 3 – HBA: one ATTO vs. two Broadcom

On the 102-drive SAS3 platform we put a single 16-port ATTO ESAH-12F0-GT0 against two Broadcom 9500-8e cards. One card in one slot read 14,705 MB/s versus 13,951 MB/s for the pair (+5.4%) and matched them on writes (11,593 vs. 11,541 MB/s, +0.4%). The headline isn’t the percentage — it’s that one ATTO card in one slot equals or beats two Broadcom cards at the SAS3 ceiling, freeing a PCIe slot and simplifying the bill of materials.

The bonus: software is the biggest lever

The largest gain we measured all year wasn’t a new drive or a faster cable — it was how LeilFS decides which zone to write next. The Stable branch “spreads” each new chunk into a different zone, which keeps fragmentation near zero and makes sequential reads excellent, but pushes writes up whenever the next operation isn’t a clean append. The Experimental branch “fills one zone” at a time, with a client flush hint (pwritev) enabling larger batched writes, so single-writer ordering keeps writes strictly sequential at the drive. On identical WD 30 TB ePMR hardware (EC6+2, 8 drives), the Experimental branch held ~1.52 GB/s writes against Stable’s ~965 MB/s  – a +57% write gain  – for a 7% read trade-off (1.52 vs. 1.63 GB/s at 32 jobs). For write-heavy archive ingest, that trade is an easy yes. Same drives, same chassis, same cables: the only thing that changed was the software’s zone-assignment strategy.

The biggest single-day performance gain we measured this year wasn’t a new drive. It was a decision in software.

That is also why “hardware-agnostic” storage leaves money on the table. In our customer PoC (Q1 2026, 102-drive WD Data102), LeilFS 5.x sustained 99.7% of raw read efficiency (14.2 GB/s vs. 14.236 GB/s raw) and 95.4% of raw write efficiency (8.278 vs. 8.679 GB/s) at 102 jobs. You only capture numbers like that with a stack that speaks the drive’s native physics.

Methodology (for the engineers)

We publish our test setups so anyone can reproduce them. The LeilFS sweep is a parameterized job-count loop across the chunkserver pool — large per-job sizes at low concurrency, smaller sizes at high concurrency, repeated for both read and write:

# Low job counts — large per-job size
for jobs in 1 2 4 8 10; do
  sudo fio --name=test${jobs} --directory=$MOUNT \
    --size=64G --rw=write --numjobs=${jobs} \
    --ioengine=libaio --group_reporting \
    --bs=1M --direct=1 --iodepth=1
  sleep 5
done

# High job counts — smaller per-job size
for jobs in 20 30 40 60 80 102; do
  sudo fio --name=test${jobs} --directory=$MOUNT \
    --size=16G --rw=write --numjobs=${jobs} \
    --ioengine=libaio --group_reporting \
    --bs=1M --direct=1 --iodepth=1
  sleep 5
done   # repeat with --rw=read for the read sweep

The raw single-drive comparison runs one FIO process per drive in ZBD mode, with offsets that skip the conventional random zones and job offsets distributed across the disk by zone position — stressing exactly the multi-stream, multi-zone load an EC chunkserver generates:

[global]
rw=write            # also rw=read for the read sweep
bs=1M
iodepth=1
direct=1
ioengine=libaio
group_reporting
size=10G
zonemode=zbd        # SMR-only; remove for CMR
offset=260113956864 # skip conventional random zones
# one fio per drive, two NUMA nodes in parallel, two reps per job count
numactl --cpunodebind=0 --membind=0 fio "$profile1" &
numactl --cpunodebind=1 --membind=1 fio "$profile2" &

The non-default chunkserver settings that matter for reproducibility: GARBAGE_COLLECTION_FREQ_MS=0, PERFORM_FSYNC=0, HDD CRC checks off for the benchmark, NR_OF_HDD_WORKERS_PER_NETWORK_WORKER=4, MAX_BLOCKS_PER_HDD_READ_JOB=1024, and WRITE_BUFFERING_SIZE_MB=64 (the last is what the Experimental zone-fill path leans on).

Three takeaways

• Choose your SMR technology deliberately. HAMR beat ePMR by +50% on writes and +9% on reads at 30 TB — but only a zone-aware stack converts that capacity into near-native HDD performance.
  
• Standardize on SAS4. +14–16% throughput over SAS3 at 60 drives, for free, on the same sizing. There is no reason to spec 12G into a new high-density build.
  
• The file system is your biggest lever — don’t stay hardware-agnostic. +57% write throughput from zone-assignment tuning alone, on identical hardware, for a 7% read trade. No drive swap comes close.

An open invitation

We publish our test benches, our erasure-coding schemes, and our raw numbers because the SMR conversation has been run on assumptions for too long. If you build storage software, run the same sweep and post your results – no NDAs, no selective disclosure. SMR isn’t the bottleneck. The software layer between the drive and the workload is. The drives are ready; the only question is whether your stack was designed for their physics or retrofitted to it.

Author: David Gerstein
David Gerstein is the CTO of Leil, an Estonia-based storage infrastructure company building HDD-native software for the exabyte era. Leil is in active co-engineering partnership with Western Digital, validated across the WD, Seagate, and Toshiba drive ecosystems.