Package: ong-utils
Ong_UtilsDescriptionSimple package with some utils to import in any project:class to manage configuration files in yaml or jsonRead more. It also useskeyringto store and retrieve passwords.Read morelogger and a timer to record elapsed times for optimizing some processes.Read moreacreate_pool_managerfunction to create instances of urllib3.PoolManager with retries and timeouts and checking of
https connections.Read moreaTZ_LOCALvariable with the local timezone
of the computer).anis_debuggingfunction that returns True when debugging codeato_listfunction to convert any non-list value (specifically strings to avoid iterating char by char) into a listacookies2headerthat converts cookies in dict to header field 'Cookie' for use in urllib3.Read moreaget_cookiesfunction to extract a dict of cookies from a response of a urllib3 request.Read morea class to store any data into keyring (e.g. strings, dicts...).Read morefunctions to parse html pages and extract javascript variables (such as CSRF tokens or links).Read moreClass to read/apply sensitivity labels to a file (Public, Internal...)Read morefunctions to get current user and domain.Read morefunctions for simple input dialogs with validations using tkRead morea functionfix_windows_gui_scaleto avoid blurry tkinter text elements in Windows 10 or 11.Read moreOptional dependenciesInstallingpip install ong_utils[shortcuts]:functions to create desktop shortcuts for packages installed with pip.Read moreInstallingpip install ong_utils[xlsx]:function to export a pandas dataframe into a xlsx excel sheet.Read moreInstallingpip install ong_utils[jwt]:functions to decode access tokens.Read moreInstallingpip install ong_utils[selenium]:class to manage Chrome using selenium.Read moreInstallingpip install ong_utils[office]:Classes to interact with API of Office in windows.Read moreInstallingpip install ong_utils[credentials]:function to verify password of current log in user.Read moreGeneral usageSimple example of aninit.py in a package ("mypackage") using ong_utils:importpandasaspdfromong_utilsimportOngConfig,LOCAL_TZ,OngTimer_cfg=OngConfig("mypackage")config=_cfg.configlogger=_cfg.loggerAnd usage in other parts of the code of "mypackage":importpandasaspdfrommypackageimportconfig,logger,LOCAL_TZ,httplocal_now=pd.Timestamp.now(tz=LOCAL_TZ)res=http.request("GET",config('url'))logger.info("Sample log")Configuration filesConfig files are yaml/json files located (by default) in~/.config/ongpi/{project_name}.{extension}.
The file extension can be yaml, yml, json or js.
File can have this form:my_project:sample_key1:sample_value1sample_key2:sample_value2log:optional_log_config:values# This is an optional section, for values used just in testsmy_project_test:sample_key1:sample_value1sample_key2:sample_value2or this{"my_project":{"sample_key1":sample_value1,"sample_key2":"sample_value2"},"log":{"optional_log_config":value}}Config files must have a section with the name the project.   OngConfig.config("key") will raise an Exception if "key" is undefined, unless
a default value is used (OngConfig.config("key", "default_value"))If a file_path is supplied as second argument to the constructor, that file will be used. In that case many projects
can share the same config file. E.g. if config file is /etc/myconfig.yaml, ti can has this form:project_name1:key_for_project1:valueproject_name2:key_for_project2:value# Optional: test values for project_name2project_name2_test:key_for_project2:valueand config method can only access to configuration of current project.New values can be added to the configuration in execution time by callingadd_app_config. That will persist the new values in the configuration file.PasswordsModule uses keyring to store passwordsfromong_utilsimportOngConfig# configuration file should have "service" and "user" keys_cfg=OngConfig("mypackage")config=_cfg.configget_password=_cfg.get_passwordset_password=_cfg.set_password# Sets password (prompts user)set_password("service","user")# Equivalent to keyring.set_password(config("service"), config("user"), input())# Gets passwordpwd=get_password("service","user")# Equivalent to keyring.get_password(config("service"), config("user"))Storing long data in keyringStoring json-serializable dataFor storing long passwords (e.g. a jwt_token) or non string data (e.g. a dictionary of cookies), useong_utils.InternalStorageclass.fromong_utilsimportInternalStorageinternal_storage=InternalStorage("your app name")forvaluetostorein["long string"*120,{"an":"example","of":"dictionary"}]:key_name="sample_key"internal_storage.store_value(key_name,value)stored=internal_storage.get_value("key name")assertvalue==storedinternal_storage.remove_stored_value()Storing cookies from requests.session objectsCookieJarobjects are not json serializable. To store cookies you'll have to turn into list of dicts.Use functions inrequests.cookiespackage to make conversions before storing/retrieving them as dicts# Assume session is a request.session.Session objectcookies=session.cookiescookie_dict=[dict(name=c.name,value=c.value,domain=c.domain,path=c.path,expires=c.expires)forcincookies]# Store cookie_dict normally# Code for updating session from cookie_dictimportrequests.cookies# Assume cookie_dict is the same as abovecookies=[requests.cookies.create_cookie(**c)forcincookies_dict]forcookieincookies:session.cookies.set_cookie(cookie)TimersOngTimerclass usestic(msg)to start timer andtoc(msg)to stop timer and show a message with the elapsed time.
Several timers can be created with differentmsg. The parametermsgis used to link methodstic(msg)andtoc(msg),
so the time is measured from tic to toc.Before a toc there must be a tic, so if there is not atic(msg)with the samemsgas atoc(msg)an exception is risen.
Additionally, it can be used as a context managerExample Usage:fromong_utilsimportOngTimerfromtimeimportsleep########################################################################################################## Standard use (defining an instance and using tic, toc and toc_loop methods, changing decimal places)#########################################################################################################tic=OngTimer()# if used OngTimer(False), all prints would be disabledmore_precise_tic=OngTimer(decimal_places=6)# Use decimals parameter to increase decimals (defaults to 3)tic.tic("Starting")more_precise_tic.tic("Starting (6 decimals)")foriinrange(10):tic.tic("Without loop")sleep(0.15)tic.toc("Without loop")tic.tic("Loop")sleep(0.1)ifi!=5:tic.toc_loop("Loop")# Will print elapsed time up to iter #5else:tic.toc("Loop")# Will print in this casesleep(1)tic.print_loop("Loop")# Forces print In any case it would be printed in destruction of tic instancetic.toc("Starting")# Will print total time of the whole loopmore_precise_tic.toc("Starting (6 decimals)")# Will print total time with 6 decimals######################################################################################### Using toc/toc_loop with a non previously defined msg will raise a ValueError Exception########################################################################################try:tic.toc("This msg has not been defined in a previous tick so ValueError Exception will be risen")exceptValueErrorasve:print(ve)############################################################## Use as a context manager. Won't work accumulating in a loop#############################################################withOngTimer(msg="Testing sleep"):print("hello context manager")sleep(0.27)withOngTimer().context_manager("Testing sleep"):# Exactly same as aboveprint("hello context manager")sleep(0.27)# Use context manager (but testing that it can be disabled)withOngTimer(msg="Testing sleep disabled",enabled=False):print("hello disabled context manager")sleep(0.22)# use global timer as context managerexisting_instance=OngTimer()withexisting_instance.context_manager("Example using an existing context manager instance"):sleep(.19)# Optionally: write also tick using a loggerimportlogginglogging.basicConfig(level=logging.DEBUG)withOngTimer(msg="Using a logger",logger=logging,log_level=logging.DEBUG):sleep(0.2)############################################################### When a timer is deleted, any tic without toc will be printed##############################################################forgoten_toc_timer=OngTimer()# This timer will have tics without corresponding tocstandard_timer=OngTimer(decimals=6)forgoten_toc_timer_disabled=OngTimer(enabled=False)forgoten_toc_timer.tic("forgotten timer1")forgoten_toc_timer.tic("forgotten timer2")standard_timer.tic("unforgotten timer")forgoten_toc_timer_disabled.tic("forgotten disabled timer")sleep(0.1)standard_timer.toc("unforgotten timer")delforgoten_toc_timer# Will print elapsed time, as are pending tocsdelstandard_timer# Prints nothing (as there is not pending tic)delforgoten_toc_timer_disabled# Prints nothing (is disabled)###################################################### Use .msgs property to iterate over all named timers#####################################################loop_timer=OngTimer()for_inrange(10):loop_timer.tic("hello1")loop_timer.tic("hello2")sleep(0.1)loop_timer.toc_loop("hello1")loop_timer.toc_loop("hello2")formsginloop_timer.msgs:loop_timer.print_loop(msg)Urllib3 utilsModule ong_utils.urllib3 includes simple functions to treat cookies in urllib3.Example:fromong_utilsimportcreate_pool_manager,cookies2header,get_cookiesurl="whichevervalidurl"http=create_pool_manager()# Creates a PoolManager with retriesreq=http.request("get",url)# get cookies (as a dict)cookies=get_cookies(req)headers={"Accept":"text/html;application/json"}# append cookies to the headers dictheaders.update(cookies2header(cookies))req.http.request("get",url,headers=headers)# Using cookies from previous responseMake shortcuts for entry pointsYou can create desktop shortcuts for each entry point in the script to easily launch them in your system.You have to install optional dependencypip install ong_utils[shortcuts]NOTE: for the shortcut to work, each entry point defined in e.g.script_fileof packagepackagemust be
executable withpython -m package.script_fileThere are two ways to create shortcuts:Create shortcuts when installing with pip:valid when installing from git (e.g. pip install
git+https://github.com/someone/somerepo.git). Uses a custom postinstall
script that creates the desktop launcher and modifies the wheel file (so they can be uninstalled) after building the
wheel file from sources.Create a script and run it manually after install:valid for any other case. Create a entry_point
e.g.post_installand call it manually after installation. That scritp will create the shortcut(s) and add it/them
to
theRECORDfile so the shortcut will be later uninstalledCreate the shortcut when installing with PIPCreate asetup.pyin your root directory and add the following code:fromsetuptoolsimportsetupfromong_utils.desktop_shortcutimportPipCreateShortcutsetup(cmdclass={'bdist_wheel':PipCreateShortcut})In yourpyproject.tomladd the following:[build-system]requires=["setuptools","wheel","ong_utils[shortcuts]"][project.scripts]script1="package.file:function"Then the program will install the wheel from pip and create a desktop shortcut for script1.Create a manual scriptProvided that you have the following entry point in yourpyproject.toml:[project.scripts]script1="mypackage.myscript:myfunction"You'll have to create a script in your code. Let's call itpost_install.pywith the following content:fromong_utils.desktop_shortcutimportPostInstallCreateShortcutdefmain():PostInstallCreateShortcut("your_library_name").make_shortcuts()if__name__=='__main__':main()Assuming thatpost_install.pyscript is in themypackagefolder, then you have to ask the user to run it manually
after installation:pipinstallmypackage
python-mmypackage.post_installNOTE: optionally, you can add icons to the shorcut with png format or icns
format (for mac), provided that the icons have the same name as the entry_point. The program will use the first icon
that matches the name of the entry point.Nicer output pandas DataFrame to ExcelYou can export a pandas DataFrame to Excel nicely formated (converted to an Excel Table, with autofilter enabled and columns widths autofitted)Example:importpandasaspdfromong_utilsimportdf_to_excelwithpd.ExcelWriter(filename)aswriter:df_to_excel(df,writer,sheet_name)Decoding jwt tokensNeeds install extra packages withpip install ong_utils[jwt]Useong_utils.decode_jwt_tokento decode a jwt token into a dict.Useong_utils.decode_jwt_token_expiryto decode expiration as a datetime object.Parsing html pagesTo extract simple values without the need for BeautifulSoup, you can usefind_js_variablefromong_utilsimportfind_js_variablesource="""Imagine this is a websitevar_name1="value1"var_name2={"key1":"value2"}"""find_js_variable(source,'var_name1')# returns "value1"find_js_variable(source,"var_name",":")# returns "value2"Control webpages with seleniumInstallong_utils[selenium]to control websites with seleniumfromong_utilsimportChrome# Use it as a context managerwithChrome(block_pages="https://www.marca.com")aschrome:driver=chrome.get_driver()driver.get("https://www.google.com")driver.implicitly_wait(5)driver.get("https://www.marca.com")driver.implicitly_wait(5)# Or close driver explicitlychrome=Chrome()driver=chrome.get_driver()driver.get("www.someserver.com")chrome.quit_driver()################################ Wait for a cookie in a request###############################driver=chrome.wait_for_cookie("someserver.com","somecookie",timeout_headless=10,timeout=60)ifdriver:cookies=driver.get_cookies()################################ Wait for a certain request###############################req=chrome.wait_for_request("someserver.com","someserver.com/api/interesting_endpoint",timeout_headless=10,timeout=60)ifreq:auth=req.headers['Authorization'].split(" ")[-1]# or the same shorter...token=chrome.wait_for_auth_token("someserver.com","someserver.com/api/interesting_endpoint",timeout_headless=10,timeout=60)iftoken:do_stuff_here()Utilities for OfficeCreate instances of office programsUse classes inong_utils.office.office, to open files with Offfice API in windows.Avoids problems with cache that might happen from time to time,Sample code:fromong_utils.office.office_baseimportWordBase,ExcelBase,PowerpointBaseclassMyWord(WordBase):def__init__(self,file,logger):super().__init__(logger)self.file=self.client.Open(file)Add Sensitivy Labels to office filesUse a sample office file to apply sensitivity labels to other file, or if you know the label name, apply the label name.Works properly with Internal and Public, might not work well with Private or Restricted filesSample code;fromong_utilsimportSensitivityLabel# Case 1: you know the sensitivity label to applysl=SensitivityLabel("XXXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX")sl.apply(my_filename)# Case 2: you don't know label name, but what to clone an existing one (must be an Excel/Word/Powerpoint file)SensitivityLabel(reference_file).apply(my_file)# Case 3: get label name from a file and print itprint(label_id:=SensitivityLabel(reference_file).label_id)# label_id can be used to apply to further docs, e.g.: SensitivityLabel(label_id).apply(filename)Get current user and domainReads it from environ variablesfromong_utilsimportget_current_user,get_current_domainprint(get_current_user())# Prints current userprint(get_current_domain())# Prints current domain. Could be emptyVerify password of current log in userInstallong_utils[credentials]to check correct password for current user. Works in windows and linux/macosfromong_utilsimportverify_credentialsusername="your current username"# get it from ong_utils.get_current_user()domain="your domain, needed in windows"# For linux/macos could be empty. Get it from ong_utils.get_current_domain()password="your password goes here"ifverify_credentials(username,domain,password):print("Your password is ok")else:print("Bad password")Simple dialogsUseong_utils.simple_dialogfor build a dialog with custom controls and validations.
Useuser_domain_password_dialogfor a dialog that shows username, domain and password.Custom dialogUseong_utils.dialogto build a dialog with multiple string entry items and custom validations. You have to inform the elements of the dialog using a list ofong_utils.ui.UiFieldclassesSample code to show the following window:fromong_utilsimportsimple_dialog,get_current_domain,get_current_userfromong_utils.uiimportUiField# Optional function to validate credentials. Could be other function or Nonefromong_utilsimportverify_credentialsfield_list=[UiField(name="domain",# Key of the dict in the return dictionary and for validation functionslabel="Domain",# Name to the shown for the userdefault_value=get_current_domain(),# Default value to be usededitable=False# Not editable (by default all fields are editable)),UiField(name="username",label="User",default_value=get_current_user()),UiField(name="password",label="Password",default_value="",show="*",# Hides password by replacing with *validation_func=verify_credentials# The validation function receives values of all fields, so should accept extra **kwargs),UiField(name="server",label="Server",width=40# Use width parameter to make this Entry field longer. Use it in all to make all fields longer)]# Call the function to open the login window with custom optionsres=simple_dialog(title="Sample form",description="Show descriptive message for the user",field_list=field_list)print(res)Login dialog form for username, domain and passwordIn the case of a simple login form such asuse the following codefromong_utilsimportuser_domain_password_dialogdefvalidation(**kwargs)->bool:"""A function that receives a dict and returns boolthe dict receives the keys username, domain and password.Could use the ong_utils.verify_credentials to verify against current log in user"""iflen(kwargs['password'])>5:returnTrueelse:returnFalse# This will show a form with current username and domain (not editable) and a password (that will be shown with *)result=user_domain_password_dialog(title="your title goes here",description="A label to show context to the user",validate_password=validation,# pass None to skip validationparent=None# or the main window to use)print(result)# could be {} if user cancelled or a dict of "username", "domain", "password"Fix windows scalingAs answered inhttps://stackoverflow.com/a/43046744, in Windows 10 or 11, tk applications texts look blurry, such as:Use the functionfix_windows_gui_scaleto make it look properly like this:Some sample code (works in any OS, but will only sharpen texts in Windows):fromong_utilsimportfix_windows_gui_scalefix_windows_gui_scale()importtkinterastkfromtkinterimportttk# write your GUI code here...