Lessons from Scaling AI Powered Translation Services Using Istio - Tim Gallant & Malini Bhandaru
Channel: CNCF [Cloud Native Computing Foundation]
Date: 15/11/2023

Lessons from Scaling AI Powered Translation Services Using Istio - Tim Gallant, Lilt & Malini Bhandaru, Intel

Lilt is a global AI solution for enterprise translation and localization. Over the last few years we have felt architectural growing pains as we have scaled up our system to handle large translation workloads for customers like Intel. Our monolithic application server was slow to deploy and it was cumbersome to develop new features and APIs. We introduced Istio to start the evolution of our system to take better advantage of kubernetes and of a microservices oriented architecture. This presentation lays out how Lilt was able to utilize Istio to introduce gradual architectural changes that led to greater system stability and higher product velocity. First, we detail how we gained traction for this initiative organizationally. Next, we go into the details of how we leveraged Istio’s observability, Ingress Gateway and External Authorization functionality. Finally, we discuss our future plans to optimize our current use of Istio and adopt more features of the platform.