Virtualization as an enabler for dynamic resource allocation in HPC

Pickartz, Simon Malte; Monti, Antonello (Thesis advisor); Müller, Matthias Stefan (Thesis advisor)

1. Auflage. - Aachen : E.ON Energy Research Center, RWTH Aachen University (2019)
Book, Dissertation / PhD Thesis

In: E.ON Energy Research Center ; Ausgabe : ACS, Automation of complex power systems 65
Page(s)/Article-Nr.: 1 Online-Ressource (xiv, 2015 Seiten) : Illustrationen

Dissertation, RWTH Aachen University, 2018

Abstract

This dissertation explores the viability of virtualization techniques to address the challenges that arise from current developments in High Performance Computing (HPC). In doing so, it follows a system-level approach enabling a more flexible utilization of the resources in supercomputers. It presents a state-of-the-art analysis of virtualization in HPC and derives a set of requirements for its efficient support in this domain. These requirements build the foundation for the virtualization-aware communication stack that has been developed as part of this work. Firstly, the communication stack enables the seamless migration of Message-Passing Interface (MPI) processes in HPC clusters while meeting the performance demands of the HPC domain. And secondly, it supports locality-awareness and topology-awareness in virtual clusters. Its viability is demonstrated by taking the example of co-scheduling. In doing so, this dissertation presents an extension to the prototype co-scheduler ``Poor Man's Co-Scheduler'' (poncos). This extension has been developed to investigate the feasibility of dynamic co-scheduling based on the applications' main memory bandwidth consumption. The concepts proposed by this dissertation support a more flexible assignment of the resources. This allows cluster maintainers to improve the overall system utilization.

Institutions

• E.ON Energy Research Center [080052]
• Institute for Automation of Complex Power Systems [616310]