Unleashing storage performance, from personal systems to data centers

Linux can now push to full speed even the fastest storage. But it throws away 80% of that speed, if control is needed (e.g., to start applications quickly or to guarantee bandwidths to clients). A new I/O scheduler now enables 90-100% of the speed to be reached, while preserving full control.

Thanks to a brand new I/O stack, Linux can now push to full speed even the fastest storage devices. But what about control? In the configurations that enable full speed to be reached, control is so poor that, e.g.,:

if there is some additional I/O in the background, then applications take an impressively long time to start (even on very fast storage);

on any kind of server, an unlucky client gets practically zero throughput. It happens if the client needs random I/O to be served, but a handful of other clients happen to be triggering sequential I/O at the same time.

To address these issues, Linux does provide mechanisms to control I/O. But the best performing mechanism, available so far, easily ends up utilizing as little as 20% of the storage speed. This just wastes the benefits of the new I/O stack, and is particularly problematic with lower-end hardware.

Fortunately, there is now a new I/O scheduler, named BFQ, which endows Linux with full control on I/O, and at the same time reaches 90% to 100% of the available speed. On the downside, BFQ is currently limited to devices with a peak rate below 400 KIOPS (work is in progress to overcome this limitation).

This presentation is devoted to showing–mainly through demos–both the above personal-system to server problems, and how BFQ solves these problems.