Data-Aware and Simulation-Driven Planning of Scientific Workflows on IaaS Clouds - Centre de Calcul de l'IN2P3 Accéder directement au contenu
Article Dans Une Revue Concurrency and Computation: Practice and Experience Année : 2022

Data-Aware and Simulation-Driven Planning of Scientific Workflows on IaaS Clouds

Résumé

The promise of an easy access to a virtually unlimited number of resources makes Infrastructure as a Service Clouds a good candidate for the execution of dataintensive workflow applications composed of hundreds of computational tasks. Thanks to a careful execution planning, Workflow Management Systems can build a tailored compute infrastructure by combining a set of virtual machine instances. However, these applications usually rely on files to handle dependencies between tasks. A storage space shared by all virtual machines may become a bottleneck and badly impact the application execution time. In this paper, we propose an original data-aware planning algorithm that leverages two characteristics of a family of virtual machines instances, i.e., a large number of cores and a dedicated storage space on fast SSD drives, to improve data locality, hence reducing the amount of data transfers over the network during the execution of a workflow. We also propose a simulation-driven approach to solve a cost-performance optimization problem and correctly dimension the virtual infrastructure onto which execute a given workflow. Experiments conducted with real application workflows show the benefits of the presented algorithms. The data-aware planning leads to a clear reduction of both execution time and volume of data transferred over the network while the simulation-driven approach allows us to dimension the infrastructure in a reasonable time.
Fichier principal
Vignette du fichier
wfs_ngos.pdf (502.67 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03649286 , version 1 (22-04-2022)

Identifiants

Citer

Tchimou N'Takpé, Jean Edgard Gnimassoun, Souleymane Oumtanaga, Frédéric Suter. Data-Aware and Simulation-Driven Planning of Scientific Workflows on IaaS Clouds. Concurrency and Computation: Practice and Experience, 2022, 34 (14), pp.e6719. ⟨10.1002/cpe.6719⟩. ⟨hal-03649286⟩
56 Consultations
76 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More