How to Eliminate the I/O Bottleneck and Continuously Feed the GPU While Training in the... - Lu Qiu
Channel: The Linux Foundation
Date: 2023-05-25

How to Eliminate the I/O Bottleneck and Continuously Feed the GPU While Training in the Cloud - Lu Qiu, Alluxio

Model training is a time-consuming, data-intensive, and resource-hungry phase in machine learning, with much use of storage, CPUs, and GPUs. The data access pattern in training requires frequent I/O of a massive number of small files, such as images and audio files. With the advancement of distributed training in the cloud, it is challenging to maintain the I/O throughput to keep expensive GPUs highly utilized without waiting for access to data. The unique data access patterns and I/O challenges associated with model training compared to traditional data analytics necessitate a change in the architecture of your data platform. In this talk, Lu Qiu will introduce a new architecture to optimize I/O in the entire data pipeline and maintain the throughput required by the GPU. Also, she will share how to implement this new architecture in Kubernetes for Pytorch workloads in the public cloud.