Zlog: Distributed Shared-log for Software-Defined Storage
CORFU is a fast shared log approach that leverages flash devices. In this project we implement CORFU on Ceph and investigate the benefits of including this log abstraction into software-defined storage, including the management of shared logs across multiple storage tiers. This project is part of the Programmable Ceph effort (http://programmability.us/).
Mantle: A Programmable Metadata Load Balancer for the Ceph File System
Mantle is a programmable metadata balancer, implemented on CephFS, that separates the metadata migration policies from their mechanisms. The implementation adds hooks into the metadata servers in CephFS so that administrators can inject code to change how the metadata servers balance load across the cluster and to pull these extensions into the CephFS mainline so it can be used in production environments. This will allow the investigation and comparison of many load balancing strategies with real workloads. This project is part of the Programmable Ceph effort (http://programmability.us/).
Object Storage over Disaggregated Storage Devices (formerly: Smart Storage Devices in Genomics)
For almost 10 years the growth of genomic data has been doubling every 7 months. That makes genomics a prime example of a domain where data growth outpaces compute intensity and therefore could greatly benefit from storage system architectures that disaggregate storage device from storage servers. Almost all storage devices use the block interface command set. In this project we explore alternative command sets that will enable storage device disaggregation, i.e. the ability of attaching a storage device to multiple hosts and dynamically attaching and detaching storage devices. Alternative command sets we will investigate include key/value interfaces, the CORFU’s SLICE interface, and object interfaces. An important goal of this project is to explore storage system architecture designs with significantly more efficient data paths between storage clients and disaggregated storage devices.
Strong Consistency in Dynamic Wireless Networks to Enable Safe and Efficient Navigation of Autonomous Vehicles.
It is believed that future services like unmanned-traffic management will depend upon strong consistency for safe operation. These services will often need to be provided over lossy wireless networks with limited bandwidth, where partitions may be frequent. We propose to study the consistency requirements of distributed systems with smart storage devices over realistic networks. In particular, the distributed systems will implement algorithms that require large amounts of (dynamically changing) data that is available to all systems via a shared log. The problem of navigation of autonomous systems will serve as the prototype application of this research.
Started: Summer ’16
An Efficient C Library Implementation of Unum 2.0.
Computers excel at computation, but are bad at math, and it’s time to fix that. For those who know the limitations of computers, an enormous amount of effort must be spent working around the corner cases in IEEE floating point. And due to those who assume naive calculations produce good answers, the world risks financial and physical harm. J. Gustafson has recently developed the universal number (unum), a new architecture for computers that restores the rules of mathematics in our current systems. Computation with unums not only enables correct math, it increases information density, parallelism, and resilience. In order to make widespread adoption possible, we propose this project to develop an open-source C library that includes a framework for optimizing its efficiency on different architectures.
Memory and Storage Coordinative Lifetime Enhancement with Near-Data Computing.
High-performance servers demand both memory and storage systems to have high throughput, energy efficiency, and resiliency, along with long lifetime. Yet these requirements are often conflict with each other. Future memory and storage systems can employ a combination of cost-effective devices, such as DRAM, NVRAM, and flash, and rely on batch data transfer to sustain high bandwidth. Our goal in this research is to achieve high-performance, energy-efficient lifetime enhancement of holistic memory/storage system, including DRAM, NVRAM, and flash. We aim to enable high bandwidth and capacity utilization of data storage without lifetime degradation by exploiting software/hardware co-design.