Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: 2017-12-12-storage-developments-in-apache-mesos.md

layout: post title: "Storage developments in Apache Mesos: Interview with Jie Yu" published: true post_author: display_name: Greg Mann gravatar: 20c5c9695f0891b6b093704331238133 twitter: greggomann tags: Storage The (CSI) is a community driven effort to standardize the way that containers connect to storage, so that any compliant container platform will be able to use storage from any compliant ecosystem vendor. I recently sat down with long-time Mesos committer Jie Yu, who is participating heavily in the development of CSI, to discuss the effort that hes been leading to build CSI compatibility into Mesos. <br> <br> Greg: Before we get started, could you tell us a bit about your history with the Apache Mesos project? <br> Jie: Ive been working on Mesos for 4 years. I started at Twitter, and then came to Mesosphere 2 years ago. Ive worked on many container-related things in the codebase. This includes - among other things - networking and, more recently, storage. Greg: Could you summarize, at a high level, the direction in which youre currently taking Mesos storage support? <br> Jie: There are many storage vendors out there, and we dont want to build all this vendor-specific code ourselves. Instead, we want a system that allows storage vendors to plug into Mesos using a consistent interface. Then we dont need to maintain code for each vendor, which requires specific expertise. Some other projects ask vendors to put their code into the main codebase. This can be painful because then the vendors release cycle is tied to the cycle of the project itself. This is hard to manage if you have many vendors. Thats why we decided to go with this out-of-tree solution for storage. Greg: What use cases are you most excited to enable? <br> Jie: Historically, we didnt support global resources like EBS. We have limited support using , but there are issues with this. I think external storage is something Im really excited about with this work. Being able to plug in arbitrary vendors storage systems is

The other thing Im excited about is that we have a solution that allows a vendor to provide just a single container image for their plugin, and well take care of the rest of the work end-to-end. So it will be very simple for people to integrate their storage system with Mesos. Greg: When can we expect these features to become available? <br> Jie: The plan is to have local CSI storage support in Mesos 1.5, then external storage support in 1.6. Greg: How can community members get involved going forward? <br> Jie: Well, there are multiple avenues. For storage vendors, you can write a CSI plugin and test it with Mesos. For Mesos users, please try out this feature! And if you want to contribute to the coding, you can reach out on the . Hopefully we can organize a storage working group in the near future once the feature lands. Greg: Whats next after Mesos 1.6? <br> Jie: If we can wrap up our plans for 1.6, then well be in pretty good shape. The next thing is building a great storage ecosystem for Mesos. We can reach out to vendors and try to get them to integrate. Greg: Is there anything else youd like to share with the community? <br> Jie: One thing to highlight is that while were doing storage work right now, the abstractions weve built for storage can be used for other purposes in Mesos - for example, they could be used for other like IP addresses. In the cloud, you could use this to manage things like AWSs ENI. In the future, we probably need a general interface to support arbitrary devices. The industry trend is the introduction of more and more specialized devices to support things like machine learning. We probably need a general interface for these types of devices so that Mesos can expose them in a unified way. <br> <br> To learn more about CSI work in Mesos, you can dig into the design document , and track the ongoing progress on . To contribute to the multi-platform effort to implement CSI, join the .