Event Information:

Date and time: February 23, 1:00pm-2:30pm
Contact CROSS for Zoom link

Abstract:

The challenges of managing streaming data are found everywhere today with a wide range of performance, safety, and consistency requirements. However, existing systems tend to have been architected in an era of spinning disks and low core count machines, leaving room for important optimizations.  In this talk we’ll cover how we built a new storage engine from scratch using a thread per core architecture for predictable tail latencies. We’ll dive deep into the challenges of adapting a well known protocol (Kafka-API) on top of this new storage engine and the performance gains of optimizing for modern hardware.

Bio:

Noah is a principal engineer at Vectorized working on high-level Kafka wire protocol compatibility, and low-level storage features and optimizations. Before joining Vectorized he was part of the Rook team at Red Hat integrating Ceph and Kubernetes. He was a CROSS research fellow and completed his Ph.D. in computer science at U.C. Santa Cruz in 2018.