Package: cutility
cutilityInstallationUsageutilsMeasure Execution TimeCheck Path Existenceloggersimple loggerioread write filesenvload_envdatadir handlerCleanerGeneric cleanerSimple Text cleanerPII Text cleanerAppendixGetting names_listReferencescutilityCommon utils for faster developmentInstallationYou can installcutilityusing pip:# to update to latest versionpipinstall--upgradecutilityUsageutilsMeasure Execution Timeimportcutilityascu@cu.get_exec_timedeffoo():importtimetime.sleep(1)foo()Output:Time taken to execute 'foo': 0:00:01.005044Check Path Existenceimportcutilityascub=cu.check_path_exist("./data/temp.txt")print(b)Output:Falseloggersimple loggerimportcutilityascu# Create a simple logger instancelog=cu.get_simple_logger()# Log an information messagelog.i("hello world of loggers")# also supports warning critical debug error messages# log.i, log.d, log.w, log.e, log.cOutput:[2023-12-17 02:21:03,847] - [INFO] : hello world of loggersioread write filesRead write files. Currently supports only 3 formats:textjsonyamlimportcutilityascu# Example 1: Reading and Writing a text filefile_content=cu.read_text("./data/example_r.txt")cu.write_text(file_content,"./data/example_w.txt")# Example 2: Reading and Writing a JSON filejson_data=cu.read_json("./data/example_r.json")cu.write_json(json_data,"./data/example_w.json")# Example 3: Reading and Writing a YAML fileyaml_data=cu.read_yaml("./data/example_r.yaml")cu.write_yaml(yaml_data,"./data/example_w.yaml")envload_env.env file formatPROJECT_ROOT=/path/to/src
DATA_ROOT=/path/to/data
CONFIG_PATH=/path/to/config.yml# load env variablesimportosfromcutilityimportload_envENV=load_env("./.env")print(ENV)print(os.getenv("DATA_ROOT"))print(os.getenv("PROJECT_ROOT"))print(os.getenv("CONFIG_PATH"))Output:True
/path/to/data
/path/to/src
/path/to/config.ymldatadir handlerMethod to standardize access to folders and configsWhat isproject_root?Directory that holds your src folder is yourproject_rootWhat isdata_root?Directory that holds all your data folder is yourdata_rootfromcutilityimportget_dir_handlerdirh=get_dir_handler(project_root="./",data_root="./data",verbose=True)print(dirh.get_data_root())print(dirh.get_project_root())Output:~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Setting paths:
Project Root: ./
Config Path: /path/to/config
Config Files: [list of config files]
Data Root: ./data

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~CleanerGeneric cleanerUse this snippet to collectively apply multiple cleaning functionsfromcutility.cleanersimportGenericSimpleTextCleanerfromtypingimportList,Dict,Tuple,Any,Callable# Create an instance of GenericSimpleTextCleanergtc=GenericSimpleTextCleaner()# Sample textsample_text="""Check out this link: https://example.com. 😎 #Python @user1, sample@gmail.com 123-456-7908 #testing # python"""# List of names for name replacementnames_list=["John","Doe","Jane","Smith"]# Define cleaning stepsall_cleaning_steps=[(gtc.replace_contacts,{"repl":" {{PHONE}} "}),(gtc.replace_emails,{"repl":" {{EMAIL}} "}),(gtc.replace_names,{"names_list":names_list,"repl":" {{PERSON_NAME}} "}),(gtc.clean_emojis,{}),(gtc.clean_extra_newlines,{}),(gtc.clean_extra_spaces,{}),(gtc.clean_hashtags,{}),(gtc.clean_profile_handle,{}),(gtc.clean_punctuations_except,{"exceptions":[",",".","\n","?","}","{"]}),(gtc.clean_unicode_characters,{}),(gtc.clean_web_links,{}),]# Apply text cleaning functionsoutput=gtc.apply_text_cleaning_functions(sample_text,all_cleaning_steps)# Print the original and cleaned textprint(sample_text)print()print(output)Simple Text cleanerUse this snippet to individually apply simple cleaning functions# simpler text cleanerfromcutility.cleanersimportSimpleTextCleanerasstct=stc.clean_emojis("🌟 Sed euismod justo t semper justo. 😊")print(t)PII Text cleanerUse this snippet to individually apply PII cleaning functionsfromcutility.cleanersimportPiiTextCleanerasptct=ptc.replace_emails(ptc.replace_contacts("My contact number is +1(123) 456 7890 and my email is email@company.com"))print(t)AppendixGetting names_listI have curated list of first names and last namespublic github databases and compiled it here in a github gist.references mentioned in the endUse this command to get names data.wgethttps://gist.githubusercontent.com/sagarsrc/e6c7361f9ba6a64b2c9ac5bb10f0285a/raw/fbcca7c6821e7aff285271a6ce42361bbe95cc0c/pii_names.jsonReferences[1] PII names datasets:~160k first names ~100k last namesIndian Names dataset