Package: shuffled-stats
A python library for performing inference on datasets with shuffled / unordered labels.This library includes functions for generating datasets and performing linear regression on datasets whose labels (the “y”) are shuffled with respect to the input features (the “x”). In other words, you should use this library to perform linear regression when you don’t know which measurement comes from which data point.Applications include: experiments done on an entire population of particles at once (flow cytometry), datasets shuffled to protect privacy (medical records), measurements where the ordering is unclear (signaling with identical tokens)Installation$ pip install shuffled_statsExamples (without noise)Let’s start with some simple examples. We construct some random 2-dimensional input data, and the corresponding labels. We then apply shuffled regression using theshuffled_stats.linregressfunction.importnumpyasnp,shuffled_statsnp.random.seed(1)x=np.random.normal(1,1,(100,2))#input featuresy=3*x[:,0]-7*x[:,1]#labelsnp.random.shuffle(y)#in-place shuffling of the labelsshuffled_stats.linregress(x,y)#performs shuffled linear regression>>>array([3.,-7.])The original weights, [3, -7], are recovered exactly.We can do another example with defined data points:x1x2y123257-1-2-3551021012Standard linear regression would clearly reveal that 1*x1 + 1*x2 = y. Let’s see what shuffled linear regression reveals with a permuted version of the labels:x=np.array([[1,2],[2,5],[-1,-2],[5,5],[2,10]])y=np.array([-3,10,7,3,12])shuffled_stats.linregress(x,y)#performs shuffled linear regression>>>array([1.,1.])Again, the weights are recovered exactly.Examples (with noise)np.random.seed(1)#for reproducibilityx=np.random.normal(1,1,(100,3))#input featuresx[:,0]=1#making a bias/intercept columny=4+2*x[:,1]-3*x[:,2]#labelsy=y+np.random.normal(0,.3,(100))#adding Gaussian noisew=shuffled_stats.linregress(x,y)np.round(w,2)>>>array([3.80,2.09,-2.91])We see that the recovered weights approximate the original weights (4, 2, -3), including the bias term.The library includes a function,shuffled_stats.generate_datasetto quickly generating datasets for testing. Here’s an example:np.random.seed(1)#for reproducibilityx,y,w0=shuffled_stats.generate_dataset(n=100,dim=3,bias=True,noise=0.3,mean=2)w=shuffled_stats.linregress(x,y)print(np.round(w0,2))>>>array([2.07,-1.47,-0.83])print(np.round(w,2))>>>array([1.79,1.55,-0.63])The weights are approximately recovered. We can quantify the relative error by usingshuffled_stats.error_in_weights.shuffled_stats.error_in_weights(w0,w)>>>0.13010948373615697#13% errorCan we improve performance by running three separate “trials” or “replications” of this experiment, each consisting of 100 unordered labels (within each trial, the ordering of the labels is unknown, but labels within a trial must correspond to data points from that trial)? We can test this easily with our library:np.random.seed(1)#for reproducibilityx,y,w0,groups=shuffled_stats.generate_dataset(n=300,dim=3,weights=[2.07,-1.47,-0.83],bias=True,noise=0.3,mean=2,n_groups=3)#fix weights to the same values as beforew=shuffled_stats.linregress(x,y,groups=groups)print(np.round(w,2))>>>array([2.09,-1.48,-0.83])shuffled_stats.error_in_weights(w0,w)>>>0.0099665304764283077#<1% errorThe weights are a lot closer this time!The library includes several different estimators (see paper for details). We can choose different estimators to compare results:np.random.seed(1)#for reproducibilityx,y,w0=shuffled_stats.generate_dataset(n=100,dim=3,weights=[1,1,1],noise=0.3,mean=1)#the true weights are [1,1]w=shuffled_stats.linregress(x,y,estimator='SM')print(np.round(w,2))>>>[0.980.981.03]w=shuffled_stats.linregress(x,y,estimator='LS')print(np.round(w,2))>>>[0.990.921.09]w=shuffled_stats.linregress(x,y,estimator='EMD')print(np.round(w,2))>>>[0.990.931.09]Examples (on datasets)Finally, we include methods to load datasets from .csv files (shuffled_stats.load_dataset_in_clusters) so that the performance of shuffled regression can be compared to that of, for example, ordinary least-squares, on real-world data. Here’s an example that uses theaccidents.csvdataset, from the UCI repository.fromsklearn.linear_modelimportLinearRegressionnp.random.seed(1)#for reproducibilityx,y,groups=shuffled_stats.load_dataset_in_clusters('accidents.csv',normalize=True,n_clusters=2)lr=LinearRegression(fit_intercept=False)#fit_intercept is false because x already includes a bias columnprint(lr.fit(x,y).coef_)>>>[1.02859104,0.03967381]print(shuffled_stats.linregress(x,y))>>>[1.123482160.02539006]Not bad, if I do say so myself! Feel free to explore shuffled regression and reach out to me if you have any questions!