• Dorier, Matthieu
• Antoniu, Gabriel
• Cheriere, Nathanaël

Abstract

Efficient resource utilization becomes a major concern as large-scale computing infrastructures such as supercomputers and clouds keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and cost.However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited.In this study, we model the duration of the commission operation, for which we obtain theoretical lower bounds. We then consider HDFS as a use case, and we show that our lower bound can be used to evaluate the performance of the commission algorithms. We show that the commission in HDFS can be greatly accelerated. With the highlights provided by our lower bound, we suggest improvements to speed the commission in HDFS.

Topics

• impedance spectroscopy