Package: kanonym4text
kanonym4text K-anonymity guerantee for textsK Anonymity for text (or in short - kanonym4text) is an open‑sourced library that receives a corpus from the user (dataframe) and returns a K-anonymized corpus with additional information about the anonymization process performed.
kanon4txet is designed to be easily utilized, toguaranteeanonymization at a certain level pre-defined by the user (k) while still preserving some of the text utilization properties.
This repo and package are part of our Y-Data data science final project, and we would love to hear your feedback and learn from it!OverviewIn this project, we aim to apply data science techniques to anonymize textual data while preserving their utility. K-Anonymity is a technique used to ensure that an individual in a dataset cannot be identified by linking their attributes to external information, by forcing each row to be identical to k-1 other rows The anonymized data can be used for various purposes like data sharing, research, and analysis without compromising privacy. We plan on creating a novel algorithm for k-anonymity. Specifically, we address the case of unstructured data items, such as texts. Using various NLP techniques, from classical to modern DL-based solutions, and testing the utility of the anonymized data.
We have tested the library on two main datasets:Amazon dataset,Enron emails datasetWe show it is able to generate anonymized corpora in both cases.Package High-Level Algorithm:Data Preprocessing: tokenization, lemmatization, and stop word removal.K-Anonymization: Generalization and Reduction.Evaluation: Utilization is evaluated by embedding distance and semantic scores.Visualization: dataset properties prior to and post anonymization will be visualized (optional)Getting StartedYou can get started with kanonym4text immediately by installing it with pip:download packagepip install kanonym4textStep 1 - creat a data frame from your corpusThe code receives a data frame containing the corpus in the following format:
"txt" - column with the texts (default column name)Step 2 - import the following and initializethe objectUsegensim word embedding modelfor you choice (default -glove-twitter-25)from kanonym4text import Kanonym
kan = Kanonym()Step 3 - read the df:import pandas as pd
df = pd.read_csv('YOUR_FILE.csv')Step 4 -  run the anonymization process on your corpusdfa, dist = kan.anonymize(df, k=2)Running instructions:The main function is calledanonymize.It's input parameters are:df- Input Dataframek- kcol- The column in df that holds the text to anonymize. Default - txtnum_stop- Number of stop word to use. Default - 1000num_jobs- Number of CPUs to utilize. Default - 1. All CPUs - -1.verbose- Output text level. Default - 0.The function outputs are:A tuple with these items:A dataframe - the same df the user sent with additional columns:general_txt - text after "generalization"anon_txt_history - changes performed on text during annonymization process:[] - replaced{} - Lemmatize() - protected word (stop-word)anon_txt - resulted anonymized textneighbors - indeces of k neigbors (Bow anonymized)An averaged cosine distance of sentence embedding for the documents before and after the anonymizaionRunning Exampleuse the following link to run some examples of the package on your own dataframe:https://colab.research.google.com/drive/1eMSSvBxtsNFMOvKrUXgbsx1g3KOQD56s#scrollTo=ci2qjboGCt0A&uniqifier=1or use the following code:from kanonym4text import Kanonym
import pandas as pd

df = pd.read_csv('YOUR_FILE.csv')

kan = Kanonym('glove-twitter-25') # creating an object from class
dfa, dist = kan.anonymize(df, k=k, n_jobs=-1, plot=True)SupportCreate a Bug ReportIf you see an error message or run into an issue, please create a bug report. This effort is valued and helps all users.Submit a Feature RequestIf you have an idea, or you're missing a capability that would make development easier and more robust, please Submit a feature request.If a similar feature request already exists, don't forget to leave a "+1". If you add some more information such as your thoughts and vision about the feature, your comments will be embraced warmly :)ContributingKanon4txt is an open-source project. We are committed to a fully transparent development process and highly appreciate any contributions. Whether you are helping us fix bugs, proposing new features, improving our documentation, or spreading the word - we would love to have you!