How Did the Model Change? Efficiently Assessing Machine Learning API Shifts 

1 INTRODUCTION . Machine learning ( ML ) prediction APIs have made it dramatically easier to build ML applications . For example , one can use Microsoft text API ( Mic , a ) to determine the polarity of a text review written by a customer , or Google speech API ( Goo , b ) to recognize users ’ spoken commands received by a smart home device . These APIs have been gaining in popularity ( MLa ; Chen et al. , 2020 ) , as they eliminate the need for ML users to train their own models . Monitoring and assessing the performance of these third-party ML APIs over time , however , is under-explored . ML API providers continuously collect new data or change their model architectures ( Qi et al. , 2020 ) to update their services , which could silently help or harm downstream applications ’ performance . For example , as shown in Figure 1 ( a ) and ( b ) , we observe a 7 % overall accuracy drop of IBM speech API on the AUDIOMNST dataset in March 2021 compared March 2020 . In our systematic study of 36 API and dataset combinations , there are 12 cases where the API ’ s performance changed by more than 1 % on the same dataset from 2020 to 2021 ( sometimes for the worse ) . Such performance shifts are of serious concern not only because of potential disruptions to downstream tasks , but also because consistency is often required for audits and oversight . Therefore , it is important to precisely assess shifts in an API ’ s predictions over time . Moreover , in this assessment , it is often more informative to quantify how the entire confusion matrix of the API has changed rather than just the overall accuracy . In the IBM case in Figure 1 , it is interesting that a major culprit of the drop in performance is the 2021 model mistaking “ four ” for “ five ” . In other settings , changes in the confusion matrix could still cause issues even if the overall accuracy stays the same . In this paper , we formalize the problem of assessing API shifts as estimating changes in the confusion matrix on the same dataset in a sample-efficient manner ( i.e. , with few calls to the API itself to minimize dollar cost ) . The straightforward approach is to compare the API ’ s prediction on randomly sampled data . However , this can require a large number of API calls to estimate the confusion matrix , which is expensive given that each call costs money . To help address this challenge , we propose MASA , a principled algorithm for ML API shift assessments . MASA efficiently estimates shifts in the API ’ s confusion matrix by clustering the dataset and adaptively sampling data from different clusters to query the API . MASA automates its sampling rate from different data clusters based on the uncertainty in the confusion matrix estimation . For example , MASA may query the ML API on more samples with the true label “ four ” than “ one ” , if it is less sure about the estimated performance change on the former . Employing an upper-confidence-bound approach to estimate the uncertainties , MASA enjoys a low computation and space cost as well as a fast estimation error rate guarantee . MASA ’ s adaptive sampling substantially improves the quality of estimation for API shifts . In extensive experiments on real world ML APIs , MASA ’ s assessment error is often an order of magnitude smaller than that of standard uniform sampling with same sample size ( e.g. , Figure 1 ( c ) ) . To reach the same tolerable target estimation error , MASA can reduce the required sample size by more than 50 % , sometimes up to 90 % . Contributions . In short , our main contributions include : 1 . We demonstrate that commercial Ml APIs can experience significant performance shifts over time , and formulate ML API shift assessments via confusion matrix difference estimation as an important practical problem . We will release the dataset consisting of 1,224,278 samples annotated by ML APIs in different date to stimulate more research on ML API shifts . 2 . We propose MASA , an algorithm to assess the ML API performance shifts efficiently . MASA adaptively determines querying the ML API on which data points to minimize the shift estimation error under a sample size constraint . We show that MASA enjoys a low computation cost and performance guarantees . 3 . We evaluate MASA on real world APIs from Google , Microsoft , Amazon and other providers for tasks including speech recognition , sentiment analysis , and facial emotion recognition . MASA leads to estimation errors an order of magnitude smaller than standard uniform sampling using the same sample size , or over 90 % fewer samples to reach the same tolerable estimation error . Related Work . Distribution shifts in ML deployments : Performance shifts in ML systems have been observed in applications like disease diagnosis ( Lipton et al. , 2018 ) , facial recognition ( Wang et al. , 2020 ) , and Inference of molecular structure ( Koh et al. , 2020 ) . Most of them are attributed to distribution shifts , i.e. , the distribution of the test and training datasets are different . Distribution shifts are usually modeled as covariate shifts ( Shimodaira , 2000 ; Sugiyama et al. , 2007 ; QuiñoneroCandela et al. , 2009 ) or label shifts ( Lipton et al. , 2018 ; Saerens et al. , 2002 ; Azizzadenesheli et al. , 2019 ; Zhao et al. , 2021 ) . API shifts are orthogonal to distribution shifts : instead of attributing the performance shifts to data distribution changes , API shifts concern with ML APIs changes which changes its predictions on the same dataset . The methods for detecting distribution drifts typically rely on changes in data feature statistics and can not detect changes in the API on the same data . To the best of our knowledge , MASA is the first work to systematically investigate ML API shifts . Deploying and monitoring ML APIs : Several issues in deployed ML APIs have been studied . For example , Buolamwini & Gebru ( 2018 ) showed that strong biases toward minority can exist in commercial APIs and Ribeiro et al . ( 2020 ) revealed that several bugs in commercial APIs can be detected using checklists . Kang et al . ( 2020 ) extend program assertions to monitor and improve deployed ML models . Chen et al . ( 2020 ) consider the trade-offs between accuracy performance and cost via exploiting multiple APIs . On the other hand , the proposed MASA focuses on estimating ( silent ) API performance changes cheaply and accurately , which has not been studied before . Stratified sampling and multi-arm bandits : Stratified sampling has proved to be useful in various domains , such as approximate query processing ( Chaudhuri et al. , 2007 ) , population mean estimation ( Carpentier & Munos , 2011 ; Carpentier et al. , 2015 ) , and complex integration problems ( Leprêtre et al. , 2017 ) . A common approach is to model stratified sampling as a multi-arm bandit ( MAB ) problem : view each data partition as an arm , and set the regret as the variance of the obtained estimator . Plenty of algorithms exist for solving standard MAB problems , such as the epsilon greedy algorithm ( Slivkins , 2019 ) , upper-confidence-bound approach ( Auer et al. , 2002 ) , and Thompson sampling method ( Russo et al. , 2018 ) . While those algorithms aim at playing the best arm as often as possible , stratified sampling ’ s goal is often to estimate the average of all arms , and thus needs to allocate the number of calls for all arms aptly based on their variance . In contrast to estimating the population average in standard stratified sampling , our goal is to simultaneously estimate a matrix whose entries can be correlated . Thus , we have an objective ( uncertainty score ) that is different from theirs ( variance ) , which also leads to an optimal allocation different from that of standard stratified sampling . This difference requires additional statistical bounds to prove convergence , which we provide in our paper . It also results in a different upper-confidence-bound term and convergence rate compared to standard bandit algorithms . 2 THE API SHIFT PROBLEM . Empirical assessment of ML API shifts . We start by making an interesting observation : Commercial ML APIs ’ performance can change substantial over time on the same datasets . We investigated twelve standard datasets across three different tasks , namely , YELP ( Dat , c ) , IMDB ( Maas et al . ) , WAIMAI ( Dat , b ) , SHOP ( Dat , a ) for sentiment analysis , FER+ ( Goodfellow et al. , 2015 ) , RAFDB ( Li et al . ) , EXPW ( Zhang et al . ) , AFNET ( Mollahosseini et al. , 2019 ) for facial emotion recognition , and DIGIT ( Dat , d ) , AMNIST ( Becker et al. , 2018 ) , CMD ( Warden , 2018 ) , FLUENT ( Lugosch et al . ) , for speech recognition . For each dataset , we evaluated three commercial ML APIs ’ accuracy in April 2020 and April 2021 . Figure 2 summarizes the overall accuracy changes . There are several interesting empirical findings . First , API performance changes are quite common . In fact , as shown in Figure 2 , API performance changes exceeding 1 % occurred in about 33 % of all ( 36 ) considered ML API-dataset combinations . Since the data distribution remains fixed , such a change is due to ML APIs ’ updates . Second , the API updates can either help or hurt the accuracy performance depending on the datasets . For example , as shown in Figure 2 ( a ) , the Amazon sentiment analysis API ’ s accuracy increases on YELP , WAIMAI , and SHOP , but decreases on IMDB . In addition , the update of Microsoft facial emotion recognition API only affects performance on the FER+ dataset , as shown in Figure 2 ( b ) . Another interesting finding is that the magnitude of the performance change can be quite different . In fact , most of the accuracy differences are between 1–3 % , but on DIGIT dataset , Google ’ s accuracy change is more than 20 % . Fine-grained assessment of API shift as changes in the confusion matrix . Based on feedback from practitioners , accuracy change alone is insufficient , and attribution to per class change is often much more informative ( Tsipras et al. , 2020 ; Hou et al. , 2019 ; Luque et al. , 2019 ) . Thus , a natural idea is to quantify an ML API ’ s performance by its confusion matrix . We assess the change of the confusion matrix over time as a measure of API shift . Formally , consider an ML service for a classification task with L labels . For a data point x from some domain X , let ŷ ( x ) ∈ [ L ] denote its predicted label on x , and y ( x ) be the true label . For example , for sentiment analysis , x is a text paragraph , and the task is to predict if the polarity of x is positive or negative . Here L = 2 , and ŷ ( x ) = 1 implies positive predicted label while y ( x ) = 2 indicates negative true label . The confusion matrix is denoted by C ∈ RL×L where C i , j , Pr [ y ( x ) = i , ŷ ( x ) = j ] . Given a confusion matrix of the ML API measured previously ( say , a few months ago ) , Co , the ML API shift is defined as ∆C , C −CO . Using confusion matrix difference to quantify the ML API shift is informative . E.g , the overall accuracy change is simply the trace of ∆C . It also explains which label gets harder or easier for the updated API . Still consider , e.g. , sentiment analysis . Given a 2 % overall accuracy change , ∆C1,2 = 1 % and ∆C2,1 = −3 % implies that the change is due to predicting less ( -3 % ) negative texts as positive , by sacrificing the accuracy on positive texts slightly ( 1 % ) . This suggests that the API could have been updated with more negative training texts .

Summary: This paper considers the problem of estimating the change in the performance of commercial ML APIs (ML as a service) as the models are updated over time (experiments are for 2020 vs 2021). It formalizes the problem as estimating the change in the confusion matrix over time. The main theoretical contribution is an adaptive sampling method to more efficiently estimate this shift. Interesting empirical results on various ML APIs are provided in the paper, showing the relevance of the problem and the effectiveness of the proposed method.