Package: concept-erasure
Least-Squares Concept Erasure (LEACE)Concept erasure aims to remove specified features from a representation. It can be used to improve fairness (e.g. preventing a classifier from using gender or race) and interpretability (e.g. removing a concept to observe changes in model behavior). This is the repo forLEAst-squares Concept Erasure (LEACE), a closed-form method which provably prevents all linear classifiers from detecting a concept while inflicting the least possible damage to the representation. You can check out the paperhere.InstallationWe require Python 3.10 or later. You can install the package from PyPI:pipinstallconcept-erasureUsageThe two main classes in this repo areLeaceFitterandLeaceEraser.LeaceFitterkeeps track of the covariance and cross-covariance statistics needed to compute the LEACE erasure function. These statistics can be updated in an incremental fashion withLeaceFitter.update(). The erasure function is lazily computed when the.eraserproperty is accessed. This class uses O(d2) memory, wheredis the dimensionality of the representation, so you may want to discard it after computing the erasure function.LeaceEraseris a compact representation of the LEACE erasure function, using only O(dk) memory, wherekis the number of classes in the concept you're trying to erase (or equivalently, thedimensionalityof the concept if it's not categorical).Batch usageIn most cases, you probably have a batch of feature vectorsXand concept labelsZand want to erase the concept fromX. The easiest way to do this is by using theLeaceEraser.fit()convenience method:importtorchfromsklearn.datasetsimportmake_classificationfromsklearn.linear_modelimportLogisticRegressionfromconcept_erasureimportLeaceErasern,d,k=2048,128,2X,Y=make_classification(n_samples=n,n_features=d,n_classes=k,random_state=42,)X_t=torch.from_numpy(X)Y_t=torch.from_numpy(Y)# Logistic regression does learn something before concept erasurereal_lr=LogisticRegression(max_iter=1000).fit(X,Y)beta=torch.from_numpy(real_lr.coef_)assertbeta.norm(p=torch.inf)>0.1eraser=LeaceEraser.fit(X_t,Y_t)X_=eraser(X_t)# But learns nothing afternull_lr=LogisticRegression(max_iter=1000,tol=0.0).fit(X_.numpy(),Y)beta=torch.from_numpy(null_lr.coef_)assertbeta.norm(p=torch.inf)<1e-4Streaming usageIf you have astreamof data, you can useLeaceFitter.update()to update the statistics. This is useful if you have a large dataset and want to avoid storing it all in memory.fromconcept_erasureimportLeaceFitterfromsklearn.datasetsimportmake_classificationimporttorchn,d,k=2048,128,2X,Y=make_classification(n_samples=n,n_features=d,n_classes=k,random_state=42,)X_t=torch.from_numpy(X)Y_t=torch.from_numpy(Y)fitter=LeaceFitter(d,1,dtype=X_t.dtype)# Compute cross-covariance matrix using batched updatesforx,yinzip(X_t.chunk(2),Y_t.chunk(2)):fitter.update(x,y)# Erase the concept from the datax_=fitter.eraser(X_t[0])Paper replicationScripts used to generate the part-of-speech tags for the concept scrubbing experiments can be found inthis repo. We plan to upload the tagged datasets to the HuggingFace Hub shortly.Concept scrubbingThe concept scrubbing code is a bit messy right now, and will probably be refactored soon. We found it necessary to write bespoke implementations for different HuggingFace model families. So far we've implemented LLaMA and GPT-NeoX. These can be found in theconcept_erasure.scrubbingsubmodule.