Afficher la notice abrégée

Sizing and Partitioning Strategies for Burst-Buffers to Reduce IO Contention

hal.structure.identifierTopology-Aware System-Scale Data Management for High-Performance Computing [TADAAM]
dc.contributor.authorAUPY, Guillaume
hal.structure.identifierHigh-End Parallel Algorithms for Challenging Numerical Simulations [HiePACS]
dc.contributor.authorBEAUMONT, Olivier
hal.structure.identifierReformulations based algorithms for Combinatorial Optimization [Realopt]
dc.contributor.authorEYRAUD-DUBOIS, Lionel
dc.date.accessioned2024-04-04T02:57:52Z
dc.date.available2024-04-04T02:57:52Z
dc.date.conference2019-05-20
dc.identifier.urihttps://oskar-bordeaux.fr/handle/20.500.12278/192602
dc.description.abstractEnBurst-Buffers are high throughput and small size storage which are being used as an intermediate storage between the PFS (Parallel File System) and the computational nodes of modern HPC systems. They can allow to hinder to contention to the PFS, a shared resource whose read and write performance increase slower than processing power in HPC systems. A second usage is to accelerate data transfers and to hide the latency to the PFS. In this paper, we concentrate on the first usage. We propose a model for Burst-Buffers and application transfers. We consider the problem of dimensioning and sharing the Burst-Buffers between several applications. This dimensioning can be done either dynamically or statically. The dynamic allocation considers that any application can use any available portion of the Burst-Buffers. The static allocation considers that when a new application enters the system, it is assigned some portion of the Burst-Buffers, which cannot be used by the other applications until that application leaves the system and its data is purged from it. We show that the general sharing problem to guarantee fair performance for all applications is an NP-Complete problem. We propose a polynomial time algorithms for the special case of finding the optimal buffer size such that no application is slowed down due to PFS contention, both in the static and dynamic cases. Finally, we provide evaluations of our algorithms in realistic settings. We use those to discuss how to minimize the overhead of the static allocation of buffers compared to the dynamic allocation.
dc.description.sponsorshipOrdonnancement de données pour le calcul haute-performance - ANR-17-CE25-0004
dc.description.sponsorshipInitiative d'excellence de l'Université de Bordeaux - ANR-10-IDEX-0003
dc.language.isoen
dc.title.enSizing and Partitioning Strategies for Burst-Buffers to Reduce IO Contention
dc.typeCommunication dans un congrès
dc.subject.halInformatique [cs]/Calcul parallèle, distribué et partagé [cs.DC]
dc.subject.halInformatique [cs]/Algorithme et structure de données [cs.DS]
bordeaux.hal.laboratoriesInstitut de Mathématiques de Bordeaux (IMB) - UMR 5251*
bordeaux.institutionUniversité de Bordeaux
bordeaux.institutionBordeaux INP
bordeaux.institutionCNRS
bordeaux.conference.titleIPDPS 2019 - 33rd IEEE International Parallel and Distributed Processing Symposium
bordeaux.countryBR
bordeaux.conference.cityRio de Janeiro
bordeaux.peerReviewedoui
hal.identifierhal-02141616
hal.version1
hal.invitednon
hal.proceedingsoui
hal.conference.end2019-05-24
hal.popularnon
hal.audienceInternationale
hal.origin.linkhttps://hal.archives-ouvertes.fr//hal-02141616v1
bordeaux.COinSctx_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.au=AUPY,%20Guillaume&BEAUMONT,%20Olivier&EYRAUD-DUBOIS,%20Lionel&rft.genre=unknown


Fichier(s) constituant ce document

FichiersTailleFormatVue

Il n'y a pas de fichiers associés à ce document.

Ce document figure dans la(les) collection(s) suivante(s)

Afficher la notice abrégée