Package: data-anonymizer
Data AnonymizerData Anonymizer is a tool that helps you anonymize data you're working with and building reports on.Installationpip3 install --user data-anonymizer** Needs Python 3 to run at the moment.Usageusage: data-anomymizer [-h] [--config CONFIG] [--delimiter DELIMITER]
                       [--generate-config] [--no-header] [--key-file KEY_FILE]
                       [--outfile OUTFILE]
                       file

positional arguments:
  file                  File to anonymize

optional arguments:
  -h, --help            show this help message and exit
  --config CONFIG, -c CONFIG
                        YAML config file (required) specifying how to
                        anonymize the data. Generate one with the --generate-
                        config flag
  --delimiter DELIMITER, -d DELIMITER
                        Specify delimiter the CSV uses (defaults to ",")
  --generate-config, -g
                        Generate config file based on CSV provided
  --no-header           Specify if a header is not present.
  --key-file KEY_FILE   Specify the file to get the key from.
  --outfile OUTFILE, -o OUTFILE
                        Name/path of file to output (defaults to anonymized-
                        INFILE_NAMEHow it worksThis tool accepts three things:A key fileIf one is not supplied, the tool generates one and saves it toanonymizer.keyor uses it if it is present in the current working directoryA config file (YAML)You can generate one for your data using--generate-configA data set to anonymize (only accepts CSV at the moment)The config file should specify how to anonymize the data in each of the CSV's columns. It should look something like this:delimiter:','columns_to_anonymize:FirstName:type:first_nameupper:trueSSN:type:ssnFullName:type:custom_nameformat:'$LAST,$FIRST$MI'DOB:type:datetimeformat:'%Y/%m/%d'range_start_date:'2000/01/01'range_end_date:'2010/01/01'The config provides metadata on the kind of data within each column. In the example above, FirstName, SSN, FullName, and DOB are the literal column headers in the CSV.Thetypeattribute in the config refers to different defined field types that data-anonymizer recognizes. All defined field types are listed below.Generating anonymous values (and maintaining referential integrity)data-anonymizergenerates anonymous values using thefakerlibrary.It's able to preserve the shape of the data and its referential integrity because of how it generates those values. It generates them like so:For every value in the data set:Concatenate value to provided key (in key file)Hash newly concatenated valueSeedfaker's random generator with the hashGenerate value for columnThis means that for a given provided key, the same value will always be converted to the same anonymous value.Say for instance that you are anonymizingsensitive-file-A.csvwith the keymy-cool-key-1234.A value ofRobert Californiain that file is anonymized toJohn Smith.If you then anonymizesensitive-file-B.csv, and it contains a record with the valueRobert California,as long as you use the same key,that record will also be anonymized toJohn Smith.Because of this, the shape of the data is preserved, and any columns that contain things that are referenced in related tables are anonymized to the same value in those tables. This preserves referential integrity.Configuration Column TypesThese are the different available types and the different attributes each recognizes. Each column type that produces output containing or potentially containing alphabetical characters also accepts the following attributes:upperSet totrueto uppercase the generated value for a given columnlowerSet totrueto lowercase the generated value for a given columncityGenerates a random real-sounding city usingfaker's city provider.customGenerate a string with a custom format. This usesfaker's syntax for itsbothifyprovider, which converts certain symbols into random letters or numbers, depending on the symbol.Special AttributesformatHere are the special symbols the format attribute recognizes:'?' : Generates a random alphabetical character'#' : Generates a random number between 0 and 9'%' : Generates a random number between 1 and 9'!' : Generates a random number between 0 and 9 or an empty string'@' : Generates a random number between 1 and 9 or an empty stringExample configuration:ID:type:customformat:'ID-??-##-%'Could yieldID-dF-03-5custom_addressGenerate address using customformatoptionSpecial AttributesformatThe format attribute for this type accepts special keywords and replaces them with generated values. It also accepts any of the special symbols thecustomcolumn configuration accepts.The special symbols thecustom_addressformatattribute recognizes are the following:$STREET: Generates a real-sounding street address (number + street name)$CITY: Generates a real-sounding city name$ZIP: Generates a valid zip code$STATE_ABBR- Selects a random abbreviated state among the 50 US states.$STATE_FULL- Selects a random spelled-out state among the 50 US states.Note:if theformatattribute is omitted, a full address is generated, including street, city, state, and zip.Example configuration:HomeAddress:type:custom_addressformat:'$STREET,$CITY,blah$STATE_ABBR#?'Could yield2073 David Square, Nixonburgh, blah AL 5gcustom_nameGenerate name using customformatoption.Special AttributesformatThe format attribute for this type accepts special keywords and replaces them with generated values. It also accepts any of the special symbols thecustomcolumn configuration accepts.The special symbols thecustom_nameformatattribute recognizes are the following:$FIRST: Generates a real-sounding first name$LAST: Generates a real-sounding last name$MI: Generates an upper-cased initialNote:if theformatattribute is omitted, a full name is generated of form$FIRST $LASTExample configuration:FullName:type:custom_nameformat:'$LAST,$FIRST$MI'Could yieldObrien, Michael FdatetimeGenerate a datetime between two dates with a custom datetime format.Special AttributesformatDesired format of the generated value (using Python datetime strptime syntax)Requiredrange_start_dateDesired start date in the range of dates to generateMust be written according to the format specified informatDefaults to: 1800/01/01range_end_dateDesired end date in the range of dates to generateMust be written according to the format specified informatDefaults to: 2018/01/01Example configuration:DOB:type:datetimeformat:'%Y/%m/%d'range_start_date:'1950/01/01'range_end_date:'2019/01/01'Could yield1976/04/23first_nameGenerates a real-sounding first name usingfaker's first_name provider.float_rangeGenerate a float between two numbers.Special AttributesstartDesired start valueRequiredendDesired end valueRequiredprecisionNumber of digits to round toExample configuration:Cost:type:float_rangestart:1end:100precision:2Could yield54.56full_addressGenerates a real-sounding full address.full_nameGenerates a real-sounding full name.int_rangeGenerate an integer between two numbers.Special AttributesstartDesired start valueRequiredendDesired end valueRequiredExample configuration:Total:type:int_rangestart:0end:1000Could yield435last_nameGenerates a real-sounding last name.optionsSelects a value from a defined list of options.Special AttributesoptionsA list of valid options to choose from.RequiredExample configuration:Sex:type:optionsoptions:-MALE-FEMALE-UNKNOWNCould yieldFEMALEssnGenerate an SSN of form###-##-####.street_addressGenerate a real-sounding street address.zipGenerate a valid-looking ZIP code.