The Best Practice of Machine Learning Platform Storage Based on CubeFS - Leon Chang, OPPO
Channel: CNCF [Cloud Native Computing Foundation]
Date: 2023-12-04

The Best Practice of Machine Learning Platform Storage Based on CubeFS - Leon Chang, OPPO

In order to meet the company's growing needs for AI training, OPPO has created a one-stop machine learning platform. With the rapid growth of business, the diversity and surge of training tasks pose challenges to storage scalability, cost, and high performance. It is difficult for the storage systems used in the early days to meet the above challenges. The speakers will share how they use the cloud-native distributed file system CubeFS to build 150PB data capacity and tens of billions of small file storage, and realize the unified storage of machine learning platforms in hybrid cloud , and support the daily training of AI business for 200 teams, 10000+ daily training tasks. They will focus on the solutions and practical experience of CubeFS's metadata management of tens of billions of small files, storage management and cache acceleration capabilities under the hybrid cloud architecture, throughout the data lifecycle to flexibly store hot and cold data.