Franck Cuny — Writing

Who owns the software?

2026-10-17T00:00:00Z

I went back to the OCP Global Summit this year, after seven years away. I wrote up what I saw in a separate report. I also came back with a question that I’ve asked myself a lot in the past: when does specialized hardware actually get adopted?

For most of my career, I saw little real interest in specialized hardware, because hardware that does something faster is useless without software support. We tried Optane at Twitter. The performance was there, the software support was not, and Intel discontinued it. We ran DPDK at Twitter and at Roblox, and operating it at scale was painful: kernel updates and testing got hard, the stack was brittle, the documentation was thin, and writing code against it hurt.

Meanwhile, the largest companies run on hardware I never touched. AWS runs its cloud on Nitro cards. Google built its own userspace networking stack. The question I want to answer is narrower: what separates the hardware they adopt from the hardware the rest of us can adopt?

Three questions instead of “boring”

Online, people answer this with “choose boring technology”. In practice, everyone calls the things they like boring and everything else an innovation token. I want criteria I can apply the same way to everything, so I use three questions.

Who owns the fix loop? When it breaks, can I (or a large community) read the code, patch it and roll it out, or do I wait for one vendor’s next release?
Does it change the contract with the software above it? Or do the OS and the application stay unaware of it?
How many people share the operating cost? Docs, tooling, people to hire, people who already hit my bug, and the odds it still exists in five years.

Kubernetes is hard to operate, and the whole industry adopted it anyway. It passes all three questions. I can patch it, a huge community shares the cost, and multiple vendors and managed offerings mean it won’t disappear. It even reduced heterogeneity: one shared model replaced a dozen homegrown deploy and scheduling systems. What matters is who carries the difficulty.

The gap starts with commodity hardware

Before specialized hardware, apply question 2 to the boring kind. A new CPU generation looks like it keeps the contract: same instruction set, same OS, same container image. It doesn’t. If my application asks for 2 cores, some instances land on Skylake and others on Sapphire Rapids. Each gets its 2 cores, and they perform very differently. Tenants all want the fastest generation, or avoid it because their code behaves differently on it, and the service owner sees the variance in their latency graphs with no easy way to explain it.

Kubernetes schedules all of this fine. The gap is the unit of resource. Kubernetes lets you label nodes, pin workloads with affinity rules, and request devices through extended resources or Dynamic Resource Allocation. All of these push the complexity between generations onto the deployment, and none of them abstract the differences away. More than ten years after its first release, the open platform still counts a core as a core.

The hyperscalers closed this gap for themselves. Google’s public Borg traces report CPU in normalized compute units, so a request means the same amount of work on any generation. That abstraction exists. It lives inside schedulers that only their owners can use.

If the open ecosystem still can’t abstract two generations of the same CPU, specialized hardware starts with a debt. It asks the platform to understand something new, on top of a gap we never closed for the old things.

What hyperscalers have that others don’t

DPUs. The hardware is familiar: a NIC with server-grade ARM cores, memory and a fast network. AWS (Nitro), Microsoft (Azure Boost) and Google (Titanium) all run parts of their infrastructure on these cards, as computers. Outside the companies that own the whole stack, most teams I know use them as a SuperNIC, where they stay transparent to the host. Using one as a computer adds a second OS per host, its own firmware and provisioning, and an ownership question: does the network team or the compute team run it?

Kernel bypass. Google built Snap, its own userspace networking stack, with its own tooling around it. DPDK is open source, so in theory anyone can patch it. It still fails the other two questions for most teams: it bypasses the kernel, so tcpdump, iptables and the observability tools we know stop working, and you need a team like Google’s to rebuild them.

Leverage with vendors. A hyperscaler gets vendor engineers on site, firmware builds for its fleet, and fixes on its own timeline. A smaller operator gets a support ticket. During an incident, my team becomes the intermediary: we relay telemetry, wait for answers, escalate, and learn that the fix is probably in the next firmware version. I don’t blame the vendor’s engineers for this. Their release cycle doesn’t follow my incident timeline, and they don’t see my fleet. They do see data from all their customers, which I never see. At OCP, Meta described sending its drive telemetry to Micron for failure prediction: each side holds half the picture, and only a customer of Meta’s size gets to combine them.

Testing the criteria

Three questions only help if they predict something. Here are cases where they do.

NVMe WRR and TP4221. NVMe has offered weighted round robin arbitration between submission queues for years. Samsung presented Linux support for it in 2017, and a patch series followed in 2020. Christoph Hellwig rejected it: WRR only controls which queue the drive fetches commands from, and says nothing about how the drive executes them. The feature existed, and nobody could define its contract with the kernel. TP4221 is the second attempt.

Optane. Intel did own much of the software path: Linux gained DAX and persistent memory support, and Intel shipped PMDK. Optane still died. It was expensive, there was only a single source of supply, and Micron’s exit from 3D XPoint mattered at least as much as the software. Owning the software is necessary, and Optane shows it doesn’t suffice. Someone also has to own enough of the workload to pay for it.

Liquid cooling. Applications don’t know their server sits on a cold plate, so it passes question 2. Operators, however, need to know about leak detection, CDU failures and facility changes, which add real operating cost. Liquid cooling spread because hyperscalers absorbed that cost first and published their designs through OCP.

Open bets

OCP 2026 also showed hardware that hasn’t faced these questions yet. I treat these as predictions, and I’ll check back.

CXL memory pools. CXL memory inside one box shows up as a NUMA node, close to transparent. Pooling across hosts needs a fabric manager, an orchestration layer and new failure modes. Samsung wired it into Kubernetes, which helps question 3, but nobody has shown how to operate it at scale.
The Open Flash Platform. Dense, low-power flash with a DPU per sled, and no answer yet to “do I run a kubelet on each sled?”.
NVIDIA CMX. NVIDIA owns the software path (Dynamo, NIXL, DOCA) and the GPU workload that needs the KV cache tier. I predict adoption, on NVIDIA’s terms.

My takeaway

Hardware gets adopted when it keeps its contract with the software above it, or when one owner controls both the software and enough of the workload to pay for it. Hyperscalers can take the second path. Everyone else depends on the open ecosystem, and that’s where the gaps are.

For teams that are not running at hyperscale, here are my recommendations:

Prefer hardware that keeps the contract, even if something with a better benchmark exists.
Ask vendors for open specs by name, like a standard telemetry format or an OCP management design.
Invest in the open layers, starting with resource models in Kubernetes, because those gaps will cost you long before any specialized hardware shows up.

The next time a vendor shows me a new piece of hardware, I won’t start with the benchmark. I’ll start with one question: who owns the software?

OCP Summit 2026: notes from the storage and memory tracks

2026-10-15T00:00:00Z

My last OCP Summit was in 2019. At the time I was building infrastructure for general compute at Twitter. Today I work on the physical infrastructure for AI inference and training, and I went to OCP Global Summit 2026 to learn. I spent most of my time in the storage track, the memory and CXL sessions, and the talks about operating AI clusters.

Four themes came out of my notes. Storage vendors now compete on endurance and host control. KV cache pushes memory out of the server. Operators want to see more and touch the workload less. And the changes that spread fastest are the ones software never notices.

Storage: endurance and control

Nobody in the storage track asked for more IOPS or more throughput. I suspect it’s a given that each PCIe generation doubles the bandwidth, and the drives follow. The questions were about endurance, the tradeoff between TLC and QLC, how much control the host gets over the drive, and reliability.

Meta gave my two favorite talks of the summit, because they spoke as operators. QLC gives them more density and better power efficiency than TLC. In exchange they get weaker sustained writes, a tight supply chain, and less endurance. At millions of drives per data center, drive failures turn into a reliability problem. Meta wants to move from reactive to proactive (predict which drive will fail) and from unplanned to planned (replace drives on their own schedule). They also want drives to treat a few bad blocks as a normal event: quarantine them and keep going. I wanted exactly this at Twitter.

The most concrete win came from data placement. By default the drive guesses where to put data, so hot, warm and cold data land in the same erase blocks. Garbage collection then copies live pages around, and write amplification goes up. With Flexible Data Placement (FDP), the application tags each write with a placement ID, and the drive keeps data with similar lifetimes together. Meta showed a large jump on their endurance metric, and a paper on FDP for CacheLib reports write amplification close to 1.

SanDisk attacked a different source of wear. QLC usually writes twice: first into an SLC cache, then into QLC. The cache eats over-provisioned space, and copying data in and out costs endurance, power and bandwidth. SanDisk’s Direct Write makes the first pass readable, so the drive no longer needs the SLC cache. The point that stayed with me: for QLC, power now limits write performance.

I also learned that SSDs carry a lot of DRAM. The drive keeps its mapping table (logical block to physical NAND location) in DRAM, roughly 1GB per TB of flash. On a 512TB drive, which can cost $250k, the DRAM adds a few thousand dollars. Across a large fleet, that’s a lot of money.

Fail in place pushes in the same direction as Meta’s wish list. The drive retires failing NAND and keeps running with less capacity. You skip a replacement, but every machine in the fleet now has a slightly different capacity and endurance, and everything above the drive has to cope with that.

The IO prioritization panel answered a question I’ve had for years. One application writes tons of logs, another reads latency-sensitive data from the same disk: how do you prioritize? Google wants high and low priority queues to the drive, with guarantees, to isolate tenants on the same host.

NVMe already tried this once: drives have offered weighted round robin (WRR) arbitration between submission queues for years, but the Linux patches were rejected because nobody could say how the drive would actually execute the priorities.

Memory leaves the server

The memory sessions all started from the same constraint: inference needs more memory, mostly for KV cache. Alibaba said a single KV cache session can reach 380GB, depending on the model and the context. Agents make it worse, because tool calls and retrieval grow the context. You can’t keep adding DIMMs to a GPU tray, so memory has to move out of the server.

CXL memory pooling started as a fix for stranded memory in general compute. The OCP Composable Memory Systems project now has AI in scope, with a fleet-level tiered KV cache as the goal: HBM, then system memory, then a shared pool.

The vendors showed very different takes. Samsung built a proof of concept with 5TB of pooled CXL memory, wired into Kubernetes: a controller exposes the pool as a resource and grows or shrinks a pod’s memory while it runs. XCENA put a DRAM cache in front of SSDs. KV cache blocks are megabytes and the access pattern is predictable, so prefetch can hide the SSD. Alibaba brought back phase-change memory, attached over CXL, with GPUs reaching it directly over UALink in the future.

The interesting part is where they disagree. XCENA’s design bets that KV cache is predictable enough to prefetch, and so does NVIDIA’s CMX, which I read about after the summit. Alibaba argued that sparse attention breaks that bet: it reads about 584 bytes per token per layer with almost no locality, so prefetch helps nothing and the tier needs low latency and small-block IOPS. So IOPS matter again, one layer up. Who is right depends on the models you serve.

The hard parts are physical. To reach a pool from the GPU you need bandwidth, which means free PCIe root ports on the host, and a GPU tray has none to spare. Removing a GPU to free them is not an option. Every connector and retimer on the way costs nanoseconds.

The best line of the track came from the CMS update: “we can’t deploy what we can’t measure.” The gap between a vendor’s benchmark and what a user has to do to reproduce it is huge. They want benchmarks that lock the whole stack, OS and libraries included, and run with a single docker compose up.

I left these talks impressed by the hardware, but without a clear idea of how I would operate it.

Operators want to see more and touch less

The OCP Optics Reliability workstream presented a telemetry spec for optical transceivers, and I could relate to every slide. Each link gets a Link Health Score, the worst of three buckets: signal quality, errors and recovery, and environment. You can query the optic at the far end of the link, and each optic keeps its own event log.

I lived this problem at Roblox. We had NICs flapping, and at first we couldn’t tell which end of the link caused it. We bisected across everything where we saw flaps (NIC models, drivers, kernels, switch models) until we pinned it to one switch model on one version of its OS. The vendor told us a newer version fixed it: the electrical signal on the port degraded slightly and caused the flap. A health score per link and telemetry from the remote end would have pointed us there much sooner.

Bare-metal hosting raised the same theme from the tenant side. The hypervisor used to be the security boundary, and the cloud provider owned everything below it. Customers who rent whole hosts now want what the hypervisor used to hide: BMC access, control over the host lifecycle, and firmware updates that don’t interrupt the workload. One proposal gives each host two BMCs, one for the provider and one isolated for the tenant. Intel showed rebootless CPU microcode updates driven from the BMC.

Meta’s other talk, on SSD telemetry, left me with more questions than answers. Meta sends SMART and NVMe logs from its drives to Micron, and Micron tells Meta which drives are about to fail. Meta knows its workloads and has years of drive data, so why does the vendor run the prediction? And what’s missing from standard telemetry to do this for any drive, from any vendor? The vendor knows its NAND, the operator knows its fleet, and neither sees the full picture alone.

From demo to adoption

At my last OCP, a few vendors showed liquid cooling. This year it was everywhere on the expo floor. The EDSFF drive form factors (E1.S, E3, E2) now include liquid cooling in their specs. Rack power is moving to high-voltage DC: NVIDIA pushes 800 VDC, and Google, Meta and Microsoft co-author Mt Diablo, a sidecar rack that moves power conversion out of the IT rack and feeds it at ±400 VDC.

None of this asks anything of the software. Applications don’t know their drive sits in a cold plate, or that the rack runs on DC. Vendors can innovate freely there, and operators adopt it without a migration plan.