Package: anonymization
AnonymizationText anonymization in many languages for python3.6+ usingFaker.InstallpipinstallanonymizationExampleReplace emails and named entities in englishThis example use NamedEntitiesAnonymizer which requirespacyand a spacy model.pipinstallspacy
python-mspacydownloaden_core_web_lg>>>fromanonymizationimportAnonymization,AnonymizerChain,EmailAnonymizer,NamedEntitiesAnonymizer>>>text="Hi John,\nthanks for you for subscribing to Superprogram, feel free to ask me any question at secret.mail@Superprogram.com\nSuperprogram the best program!">>>anon=AnonymizerChain(Anonymization('en_US'))>>>anon.add_anonymizers(EmailAnonymizer,NamedEntitiesAnonymizer('en_core_web_lg'))>>>anon.anonymize(text)'Hi Holly,\nthanks for you for subscribing to Ariel, feel free to ask me any question at shanestevenson@gmail.com\nAriel the best program!'Or make it reversible with pseudonymize:>>>fromanonymizationimportAnonymization,AnonymizerChain,EmailAnonymizer,NamedEntitiesAnonymizer>>>text="Hi John,\nthanks for you for subscribing to Superprogram, feel free to ask me any question at secret.mail@Superprogram.com\nSuperprogram the best program!">>>anon=AnonymizerChain(Anonymization('en_US'))>>>anon.add_anonymizers(EmailAnonymizer,NamedEntitiesAnonymizer('en_core_web_lg'))>>>clean_text,patch=anon.pseudonymize(text)>>>print(clean_text)'Christopher,\nthanks for you for subscribing to Audrey, feel free to ask me any question at colemanwesley@hotmail.com\nAudrey the best program!'revert_text=anon.revert(clean_text,patch)>>>print(text==revert_text)trueReplace a french phone number with a fake oneOur solution supports many languages along with their specific information formats.For example, we can generate a french phone number:>>>fromanonymizationimportAnonymization,PhoneNumberAnonymizer>>>>>>text="C'est bien le 0611223344 ton numéro ?">>>anon=Anonymization('fr_FR')>>>phoneAnonymizer=PhoneNumberAnonymizer(anon)>>>phoneAnonymizer.anonymize(text)"C'est bien le 0144939332 ton numéro ?"More examples in/examplesIncluded anonymizersFilesnamelangFilePathAnonymizer-InternetnamelangEmailAnonymizer-UriAnonymizer-MacAddressAnonymizer-Ipv4Anonymizer-Ipv6Anonymizer-Phone numbersnamelangPhoneNumberAnonymizer47+msisdnAnonymizer47+DatenamelangDateAnonymizer-OthernamelangNamedEntitiesAnonymizer7+DictionaryAnonymizer-SignatureAnonymizer7+Custom anonymizersCustom anonymizers can be easily created to fit your needs:classCustomAnonymizer():def__init__(self,anonymization:Anonymization):self.anonymization=anonymizationdefanonymize(self,text:str)->str:returnmodified_text# or replace by regex patterns in text using a faker providerreturnself.anonymization.regex_anonymizer(text,pattern,provider)# or replace all occurences using a faker providerreturnself.anonymization.replace_all(text,matchs,provider)You may also add new faker provider with the helperAnonymization.add_provider(FakerProvider)or access the faker instance directlyAnonymization.faker.BenchmarkThis module is benchmarked onsynth_datasetfrompresidio-researchand returns accuracy result(0.79) better than Microsoft's solution(0.75)You can run the benchmark using docker:dockerbuild.-f./benchmark/dockerfile-tanonbench
dockerrun-it--rm--nameanonbenchanonbenchLicenseMIT