Sug Talk: Argonne’s Ryan Chard On The Funcx Function Serving Platform For Hpc

TL;DR: HPC now serving FaaS (Functions-as-a-Service); introducing FuncX. Talk here.

SUG Series Introduction

The inaugural meeting of the Singularity User Group (SUG) was held March 12-13, 2019, at the San Diego Supercomputer Center (SDSC). The event attracted diverse representation from the international advanced computing community as conveyed through the post-event press release issued jointly by SDSC and Sylabs.

Over the course of the two-day event, over 20 talks were presented by members of the Singularity user, developer, and provider community. Because SUG generated a significant amount of interest, even from those who were unable to attend, we are sharing online each of the talks presented.

SUG Talk Introduction

Functions-as-a-Service (FaaS) are coming to HPC. Already employed in some services-oriented infrastructures, for example, Argonne Scholar Ryan Chard and collaborators are pioneering their introduction into HPC. From use of Singularity and Globus, this is an HPC-centric introduction indeed. The abstract for Ryan’s contributed talk entitled FuncX: A Function Serving Platform for HPC is as follows:

There is a growing need to support the execution of short-duration tasks on High Performance Computing (HPC) infrastructure. To address this need we have developed FuncX: a function serving platform that is designed to provide on-demand execution of stateless functions directly from existing HPC infrastructure. FuncX aims to remove the barriers that exist between functions and large-scale computing infrastructure by simplify execution of function-based workloads, enabling integration of HPC capabilities in applications and analysis pipelines, and ultimately increasing the usability of computing infrastructure. FuncX uses the Parsl scripting library to provide low-latency, scalable, and reliable execution of tasks across heterogeneous computing environments. It securely isolates function executions within containerized Singularity execution environments and leverages Globus, as an identity and access management framework and to automate the movement of functions and data. In this talk I will describe the use cases that have motivated the development of FuncX, present the design of the FuncX execution framework, and present the prototype implementation and early experiences using the system.

Ryan’s talk from SUG can be found below and here. Enjoy!