Package: kernelquantifier
Kernel QuantificationKernelQuantificationused Kernel Mean Embedding to estimate the proportion of a mixture of distributions.OverviewKernelQuantificationis a method to do quantification learning, i.e. estimate proportions of labels in a target sample, using Kernel Mean Embedding and Random Fourier Features. This python implementation can run on CPU or on GPU if cuda is installed.The methods implemented in this package are detailed in the following article:ARTICLE NOT YET AVAILABLEThe code associated to the article can be found here:CODE NOT YET AVAILABLEInstallationUse the package managerpipto install Kernel Quantifier frompypi.pipinstallkernelquantifierYou will requireCUDAto use the algorithms on your GPU.Notethat the KernelQuantifier can be use on CPU however Generative KernelQuantifier requires a GPU to be efficient.ExampleWe test our methods on a toy dataset:Iris flower datasetcontains in the scikit-learn package.All the code can be found at./tests/example.ipynb.Packageimportkernelquantifieraskqimporttorchimportnumpyasnpimportmatplotlib.pyplotaspltfromsklearn.datasetsimportload_irisfromsklearn.decompositionimportPCADataWe load the data and we use apply a PCA on the data to plot it. Code taken from thescikit-learn documentationiris=load_iris()X=iris.datay=iris.targetpca=PCA(n_components=2)X_pca=pca.fit_transform(X)colors=["navy","turquoise","darkorange"]plt.figure(figsize=(8,8))forcolor,i,target_nameinzip(colors,[0,1,2],iris.target_names):plt.scatter(X_pca[y==i,0],X_pca[y==i,1],color=color,lw=2,label=target_name,)plt.legend(loc="best",shadow=False,scatterpoints=1)plt.axis([-4,4,-1.5,1.5])plt.savefig("figures/iris_pca.png")plt.show()We split the dataset in two part, one will be the Source and the other the Target. We don't have acces to the label of the Target
dataset.np.random.seed(123)idx=np.random.choice(np.arange(X.shape[0]),75,replace=False)idx_inv=[]foriinrange(X.shape[0]):ifinotinidx:idx_inv.append(i)# TargetTarget=X[idx]Target_Label=y[idx]_,counts=np.unique(Target_Label,return_counts=True)pi=counts/Target_Label.shape[0]# SourceSource=X[idx_inv]Source_Label=y[idx_inv]# PlotSource_pca=pca.transform(Source)Target_pca=pca.transform(Target)colors=["navy","turquoise","darkorange"]plt.figure(figsize=(8,8))forcolor,i,target_nameinzip(colors,[0,1,2],iris.target_names):plt.scatter(Source_pca[Source_Label==i,0],Source_pca[Source_Label==i,1],color=color,lw=2,label=target_name,)plt.scatter(Target_pca[:,0],Target_pca[:,1],color="gray",lw=1,label="Target",)plt.legend(loc="best",shadow=False,scatterpoints=1)plt.axis([-4,4,-1.5,1.5])plt.show()Preparing the dataWe transform the target into a torch.tensor.Target=torch.from_numpy(Target)We have implemented a classkernelquantifier.LabelledCollectionto manage the Source data. This is essentially a list of tensor, where each index correspond to one class, with a few more methods implemented (see the documentation for more information).To createkernelquantifier.LabelledCollectionwe need to pass as argument a function that will transform the data as a list and the corresponding arguments (see the documentation for more information).\
For instance in our case the SourceXcome with labely. We create a functionto_labelledCollectionthat takes the data and the labels as arguments and returns a list where index 0 contains the point belonging to the class 0 and so on.Side note on the device:The computation can be done either on the CPU or the GPU. To be performed on the GPU you will needCUDAinstalled.The device the computation will be done on, is the same device as the Source (If Target and Source are not on the same device, the Target will be moved).We have implemented two functions :kernelquantifier.choose_devicethat will select the gpu is avalaible and the cpu otherwise. Andkernelquantifier.cuda_infothat will give informations on the device.device=kq.choose_device(verbose=True)# device = torch.device("cpu")kq.cuda_info(device)outputs:RunningontheGPU
Usingdevice:cuda:0

NVIDIARTXA2000LaptopGPU
MemoryUsage:
Allocated:0.0GB
Cached:0.0GB11.3defto_labelledCollection(X,y):return[torch.from_numpy(X[y==i,:]).to(device)foriinrange(3)]Source=kq.LabelledCollection(to_labelledCollection,Source,Source_Label)KernelQuantifierThere is two version of the Kernel Quantifier algorithm. The one that use Random Fourier Features and the one that don't. See notebookcomputation_times.ipynbor the article to see the difference between the two.With RandomFourierFeaturesTo use the KernelQuantifier with RFF, we call the classkernelquantifier.KernelQuantifierRFF(see the docs for more information).\
The init as two arguments: kernel_type (str) and seed (int). The seed is there to ensure the reproducibility of the experiments. While the kernel_type specify the kernel we want to use. Use the functionkernelquantifier.available_kernel_rffto see the list of currently available kernel with RFF.\
Then we have to fit our kernel. The methodKernelQuantifierRFF.fitis used to store the RFF given a sigma and the number of RFF. If sigma is a range and not a float, then the method will compute the optimal sigma according to our theoretical criterion (see the article).quantifier=kq.KernelQuantifierRFF(kernel_type="gaussian",seed=123)quantifier.fit(Source,sigma=[0.1,2],verbose=True,number_rff=1000)return:Sigma=0.9530612244897959We now can quantify using the methodKernelQuantifierRFF.quantify. This method take as argument the Source (as a LabelledCollection) and the Target. It returns the estimate proportion as annumpy.array. We print the estimated proportionspi_hatand the true proportions computed earlierpi. Finally we usedkernelquantifier.KL_divergenceto compute the KL divergence times 100 between the two vectors.pi_hat=quantifier.quantify(Source,Target)print(f"pi_hat ={pi_hat}\npi ={pi}")print(f"Error ={kq.KL_divergence(pi_hat,pi)}")outputs:pi_hat=[0.365572790.314539920.31988729]pi=[0.360.266666670.37333333]Error=0.8126347074855538Without RandomFourierFeaturesThe version without RFF is similar to the one with RFF.\
We creat an instance of the classkernelquantifier.KernelQuantifier. Use the functionkernelquantifier.available_kernelto see the list of currently available kernel without RFF. We then use the methodKernelQuantifier.fitwitout specifying the number of RFF.quantifier=kq.KernelQuantifier(kernel_type="gaussian",seed=123)quantifier.fit(Source,sigma=[0.1,2],verbose=True)We then quantify using the methodKernelQuantifier.quantifywith the same arguments in input as before.assertisinstance(Source.data_,list)pi_hat=quantifier.quantify(Source.data_,Target)# quantifier.quantify(Source, Target)print(f"pi_hat ={pi_hat}\npi ={pi}")print(f"Error ={kq.KL_divergence(pi_hat,pi)}")outputs:pi_hat=[0.366419910.31240380.32117629]pi=[0.360.266666670.37333333]Error=0.759835032840004Generative KernelQuantifierGenerative KernelQuantifier was designed to deal with the Generalised Label Shift Hypothesis. This is not the case here. We don't need to use GKQuant. Nevertheless we will use it as an example.\
We create an instance of classkernelquantifier.GenerativeKernelQuantifier. We specify the kernel_type (usekernelquantifier.available_kernel_rffto see the list of available kernel), and the generator type. Usekernelquantifier.available_generatorto see the list of available generator.\
Currently three:sharelinear: $g_i(x) = Ax + b_i$independantlinear $g_i(x) = A_i x + b_i$translation $g_i(x) = x + b_i$
$A \text{and} A_i$ are diagonal matrix.We fit the sameway we fitted thekernelquantifier.KernelQuantifier.quantifier=kq.GenerativeKernelQuantifier(kernel_type="gaussian",generator_type="sharelinear",seed=123)quantifier.fit(Source,sigma=1.,verbose=True,number_rff=1000)The method quantify has several parameters (see the pseudo-code in the article for more details and the docs).parameter={"n_epoch":20,"n_epochGM":10,"lr":0.001,"verbose":True,"initial_prop":None}pi_hat=quantifier.quantify(Source,Target,**parameter)print(f"pi_hat ={pi_hat}\npi ={pi}")print(f"Error ={kq.KL_divergence(pi_hat,pi)}")outputs:pi_hat=[0.358588630.301749210.33966216]pi=[0.360.266666670.37333333]Error=0.3781702910014444ContributingPull requests are welcome.LicenseKernelQuantifier is under theMITlicence. See the LICENCE file.