Franck Cuny

Who owns the software?

I went back to the OCP Global Summit this year, after seven years away. I wrote up what I saw in a separate report. I also came back with a question that I’ve asked myself a lot in the past: when does specialized hardware actually get adopted?

For most of my career, I saw little real interest in specialized hardware, because hardware that does something faster is useless without software support. We tried Optane at Twitter. The performance was there, the software support was not, and Intel discontinued it. We ran DPDK at Twitter and at Roblox, and operating it at scale was painful: kernel updates and testing got hard, the stack was brittle, the documentation was thin, and writing code against it hurt.

Meanwhile, the largest companies run on hardware I never touched. AWS runs its cloud on Nitro cards. Google built its own userspace networking stack. The question I want to answer is narrower: what separates the hardware they adopt from the hardware the rest of us can adopt?

Three questions instead of “boring”

Online, people answer this with “choose boring technology”. In practice, everyone calls the things they like boring and everything else an innovation token. I want criteria I can apply the same way to everything, so I use three questions.

  1. Who owns the fix loop? When it breaks, can I (or a large community) read the code, patch it and roll it out, or do I wait for one vendor’s next release?
  2. Does it change the contract with the software above it? Or do the OS and the application stay unaware of it?
  3. How many people share the operating cost? Docs, tooling, people to hire, people who already hit my bug, and the odds it still exists in five years.

Kubernetes is hard to operate, and the whole industry adopted it anyway. It passes all three questions. I can patch it, a huge community shares the cost, and multiple vendors and managed offerings mean it won’t disappear. It even reduced heterogeneity: one shared model replaced a dozen homegrown deploy and scheduling systems. What matters is who carries the difficulty.

The gap starts with commodity hardware

Before specialized hardware, apply question 2 to the boring kind. A new CPU generation looks like it keeps the contract: same instruction set, same OS, same container image. It doesn’t. If my application asks for 2 cores, some instances land on Skylake and others on Sapphire Rapids. Each gets its 2 cores, and they perform very differently. Tenants all want the fastest generation, or avoid it because their code behaves differently on it, and the service owner sees the variance in their latency graphs with no easy way to explain it.

Kubernetes schedules all of this fine. The gap is the unit of resource. Kubernetes lets you label nodes, pin workloads with affinity rules, and request devices through extended resources or Dynamic Resource Allocation. All of these push the complexity between generations onto the deployment, and none of them abstract the differences away. More than ten years after its first release, the open platform still counts a core as a core.

The hyperscalers closed this gap for themselves. Google’s public Borg traces report CPU in normalized compute units, so a request means the same amount of work on any generation. That abstraction exists. It lives inside schedulers that only their owners can use.

If the open ecosystem still can’t abstract two generations of the same CPU, specialized hardware starts with a debt. It asks the platform to understand something new, on top of a gap we never closed for the old things.

What hyperscalers have that others don’t

DPUs. The hardware is familiar: a NIC with server-grade ARM cores, memory and a fast network. AWS (Nitro), Microsoft (Azure Boost) and Google (Titanium) all run parts of their infrastructure on these cards, as computers. Outside the companies that own the whole stack, most teams I know use them as a SuperNIC, where they stay transparent to the host. Using one as a computer adds a second OS per host, its own firmware and provisioning, and an ownership question: does the network team or the compute team run it?

Kernel bypass. Google built Snap, its own userspace networking stack, with its own tooling around it. DPDK is open source, so in theory anyone can patch it. It still fails the other two questions for most teams: it bypasses the kernel, so tcpdump, iptables and the observability tools we know stop working, and you need a team like Google’s to rebuild them.

Leverage with vendors. A hyperscaler gets vendor engineers on site, firmware builds for its fleet, and fixes on its own timeline. A smaller operator gets a support ticket. During an incident, my team becomes the intermediary: we relay telemetry, wait for answers, escalate, and learn that the fix is probably in the next firmware version. I don’t blame the vendor’s engineers for this. Their release cycle doesn’t follow my incident timeline, and they don’t see my fleet. They do see data from all their customers, which I never see. At OCP, Meta described sending its drive telemetry to Micron for failure prediction: each side holds half the picture, and only a customer of Meta’s size gets to combine them.

Testing the criteria

Three questions only help if they predict something. Here are cases where they do.

NVMe WRR and TP4221. NVMe has offered weighted round robin arbitration between submission queues for years. Samsung presented Linux support for it in 2017, and a patch series followed in 2020. Christoph Hellwig rejected it: WRR only controls which queue the drive fetches commands from, and says nothing about how the drive executes them. The feature existed, and nobody could define its contract with the kernel. TP4221 is the second attempt.

Optane. Intel did own much of the software path: Linux gained DAX and persistent memory support, and Intel shipped PMDK. Optane still died. It was expensive, there was only a single source of supply, and Micron’s exit from 3D XPoint mattered at least as much as the software. Owning the software is necessary, and Optane shows it doesn’t suffice. Someone also has to own enough of the workload to pay for it.

Liquid cooling. Applications don’t know their server sits on a cold plate, so it passes question 2. Operators, however, need to know about leak detection, CDU failures and facility changes, which add real operating cost. Liquid cooling spread because hyperscalers absorbed that cost first and published their designs through OCP.

Open bets

OCP 2026 also showed hardware that hasn’t faced these questions yet. I treat these as predictions, and I’ll check back.

My takeaway

Hardware gets adopted when it keeps its contract with the software above it, or when one owner controls both the software and enough of the workload to pay for it. Hyperscalers can take the second path. Everyone else depends on the open ecosystem, and that’s where the gaps are.

For teams that are not running at hyperscale, here are my recommendations:

  1. Prefer hardware that keeps the contract, even if something with a better benchmark exists.
  2. Ask vendors for open specs by name, like a standard telemetry format or an OCP management design.
  3. Invest in the open layers, starting with resource models in Kubernetes, because those gaps will cost you long before any specialized hardware shows up.

The next time a vendor shows me a new piece of hardware, I won’t start with the benchmark. I’ll start with one question: who owns the software?

← Home